Survey Results

Thank you to both contributors and consumers for your feedback to the recent survey. It's been a busy couple of weeks with OpenNTF involved in Engage and the Business Partner Bluemix workshop from the last two days. But the "early birds" at Engage last week will have seen some high level highlights of the survey results. Here they are in a bit more depth:

Consumer Survey
There were 79 responses, which is in line with surveys a couple of years ago about Collaboration Today. That figure was the benchmark for me when setting up the survey.

1. Approximately how many OpenNTF projects (full projects, XSnippets, controls, tools) have you used in production?
Over 80% were in the middle range of 1 - 10. 11% used more than 10. Maybe there is some education we need to do, because when I consider things like OpenNTF Domino API, OpenLog, XPages OpenLog Logger, Extension Library, XPages Debug Toolbar, various XSnippets like Fredrik Norling's Standby Dialog Custom Control, Phase Listeners, DateConverters, not forgetting tools like Swiper and others, I think I'm way past 10. There is certainly more that can be done to maximise use.

2. Do you agree OpenNTF projects provide good code?
Over 90% agreed strongly or somewhat, which is encouraging.

3. Do you agree OpenNTF projects provide useful techniques to support training?
78% agreed with this. Certainly the projects that I've developed and the session databases I've created aim to train and provide learning techniques. After all, I'm usually developing them to learn something myself and, if I'm going to spend time on them, it makes sense to try to increase my knowledge at the same time. But maybe there are things we can do to share that more (both from the contributors and from consumers).

4. Do you agree OpenNTF projects are easy to install?
About two thirds agreed. Yes there is work to do here and I've heard comments from one consumer recently who had some problems with a project but will be putting together things he learned to feed back to improve the project. I think that's an encouraging and important point, that OpenNTF is not a shop for contributors to put code and consumer to use it. It's a community, and if a consumer hits problems with my project , I would want them to feed back "feature requests" for improving the documentation as much as feature requests on the project. They are projects we as a community own, not just donated code. Consumers can contribute to projects in many ways, one of which is proposing improvements on documentation or functionality to improve deployment experience, and I would hope contributors would welcome that.

5. When installing OpenNTF software, are you confident it will work on your server?
Nearly 80% agreed. This question was prompted by my own experience of struggling to install some projects. The figure of 80% is encouraging, but does mean there is still room for improvement. But the improvements can be made by OpenNTF as well as by contributors.

6. Do you agree defects on OpenNTF projects are dealt with in a manner appropriate for open source software?
About two thirds agreed, but there are many ways we can improve this and I would hope to see some during this year. This doesn't mean we'll regularly be expecting much more from contributors and expecting more significant levels of support. And we're conscious that some defects may be because a project is no longer supported for whatever reason, but there's no way for the consumer to see that. But there's no visible way for consumers to seek help other than posting to the project and waiting for a response. There's also no easy way for potential contributors to get involved or for project chefs to easily request more contributors. And only recently have projects embraced open source, allowing developers to fork the code and submit pull requests, if they are confident in doing so. There is some education to be done here, we know.

7. Do you think OpenNTF projects would benefit from larger project teams?
Nearly 85% agreed with this question. We don't want to take kudos away from contributors, but neither do we want them to feel tied to a project forever. OpenNTF is a community and open source should be managed by the community.

8. Do you find it easy to locate OpenNTF projects relevant for you?
Less than 60% agreed and that doesn't surprise us. Look for some initiatives around this and look for requests for involvement from consumers as well as contributors. After all, if you use a project, you know what it does.

9. Do you feel aware of new OpenNTF projects?
10. Which of the following would help you be more aware of new projects?
60% agreed. 50% wanted more blog posts, 21% felt weekly tweets would help, 63% wanted a newsletter and 48% monthly webinars. I only recently became aware that time was being donated to OpenNTF to do some of these areas. The interest in a newsletter was a little surprising, but after some digging into some administrative systems, it appears this used to be done. Some work needs to be done to allow people to opt in / out of newsletters, but it's something we will look at doing. I will also re-iterate that if anyone wants to blog about their project, we're more than happy to post guest blogs, so please get in touch with any of the board on various social media channels or on the Slack chat.

Comments
There was some useful feedback around usability of the site. There is definitely work required based on a completely different scope than the website redevelopment a couple of years ago. There are some difficult decisions going forward and I don't think we'll be able to have a single layout to cover older projects and yet also best support brand new ones. There may be the opportunity for new contributors to get involved, so stay tuned.
There was a comment about fixing Team MailBox. We'll take that on board and there's work to be done, but it won't be forgotten though there will not be any immediate action.

Contributor Survey
There were 55 responses. It's hard to gauge whether that's good or bad.

1. When was a release last uploaded for a project you work on?
Just over a third have had a release uploaded in the last year. That's a little disappointing, but is not surprising based on what we're seeing on releases. It also fits in with many projects being single developer projects, which means more effort required per capita per release.

2. Do you agree your projects are adequately tested?
This is admittedly a subjective question, but 78% agreed.

3. Do you agree your projects are adequately documented?
Two thirds agreed. Yes, we're developers: what we do well is develop, documentation is not our best competence. So in a single developer project, it can vary.

4. Are you involved in a multi-developer project? This includes a project that uses a pre-existing OpenNTF project or is used within another OpenNTF project.
With the second sentence in the question, I'm very much thinking of things like the Extension Library, OpenNTF, XPages Debug Toolbar (presumably included in at least one project). Only 25% agreed. This is an area we can help to do something about and avoid single point of failure on projects, support projects where the developer moves to other technologies, and turn consumers into contributors without the huge pressure of having to come up with their own idea or feel the pressure of a level of quality they may not yet feel totally comfortable with.

5. Do you feel you need help on a project you're involve in (e.g. additional developers, testers, for better documentation, someone to take over projects because you are no longer working in that area)?
45% agreed, 55% disagreed. This may not be appropriate for all projects, but may help turn consumers into contributors, help with mentoring etc. We're not saying single developer projects will be banned, but the numbers of contributions and (regular) contributors is not significant - as shown by question 1 - and to have a flourishing open source community, we need to do all we can to increase contribution and consumption.

6. Do you feel you respond in a timely manner to defects and feature requests?
Just over 70% agreed. It takes a lot of honesty to admit if you don't and we're conscious there is more we can do to support those contributors.

7. Have you received reviews or feedback of any of your projects?
About 50% have received feedback on OpenNTF, about 60% outside. There is more we can do around to increase that.

8. Would you feel confident about large organisations using your project(s)?
80% agreed, which is good to hear. Hopefully larger organisations are becoming more open to the use of open source. It's almost certain that they do in some form.

9. Do you use source control for any of your projects?
57% do, another 19% plan on doing so. Admittedly, if you've got a traditional Notes Client solution, this is difficult. This may be new for some developers, but there are some great NotesIn9 episodes covering it and Cameron Gregor's Swiper tool (also on OpenNTF) really helps.

10. Would integrated build management be of benefit to your development process (processing of test suites, automated build and releases)?
40% said yes, 22 % didn't know enough about it. Again, traditional Notes Client solutions may not benefit from this. There's also some education we need to do on the benefits and the infrastructure for it is very new, but it's something we would like to offer. Look out for more about this in the future.

Comments
One of the comments about working in groups was admitting a lack of understanding of the tools like Git, Maven, JUnit etc. We'll take that on board for webinars. We're conscious there's a lot that's new to Domino developers and one of the roles of OpenNTF is to educate on tools some of us are starting to consider better practice.
Another comment was about increasing feedback rather than just consumption, something we also feel is important.

Next Steps
The feedback we've received, combined with our own thoughts, means there is a lot of work ahead. There are some clear infrastructure and communication enhancements that the OpenNTF board should make. We need to agree on those as a board and action. Like you all, we also have day jobs, and our next board meeting is not for two weeks, so do not expect immediate announcements.

Beyond that, there are project governance enhancements that we need to discuss, draft and solicit feedback on. OpenNTF is a community so agreement needs to come as a group. We don't want to drive developers away from OpenNTF. We want to increase contribution and consumption, while promoting and valuing what OpenNTF can offer over and above open source repositories like GitHub and BitBucket, both to contributors and to consumers. Details will come out in due course (weeks not days), but if you have any concerns, please reach out to us on the Slack channels or via other social media.

Paul Withers