The question in question is the question of questions

This week we launched our tenth biannual developer survey – asking thousands of developers around the world what they’re working on and how they’re doing it. If you’re involved in software development, in any way, then go and fill it out – it will only take you fifteen minutes and I’ll wait here while you do.

Answers are easy. It’s asking the right questions which is hard.
Doctor Who – the fourth one

At VisionMobile we spend a lot of time composing questions, especially when we’re compiling a survey like this one. The process sounds easy enough – phrasing 30 questions we’d like answered to provide insight into the developer ecosystem, but it turns out to be a surprising challenge.

For a start we have to create a lot more than 30 questions: the survey tailors itself to ask each person about the industry in which they work, based in the first round of questions. The survey can thus be designed to last around fifteen minutes, but the whole industry can be covered.

Some questions we repeat every six months, choice of language, mobile platforms, and so forth, so we can spot developing trends, but sometimes an old question will need new options as the industry changes. A year ago we added Swift to our language list, and were surprised to see how quickly it had gained popularity, now we’ll be waiting to see if that growth has been sustained and at what cost.

Other questions are created from scratch: the technology behind the Internet of Things might not be entirely new, but the developer interest is. For the first time we’re asking about Open Source in IoT, drilling down to see how this new industry is evolving.

After the questions are written the task is far from over, for once the words are down then the “discussions” can begin. How many options should be listed? Which toolkits are worthy of mention? Whose products should be used as examples? How many IDEs can one developer realistically use? VisionMobile employs experts in many fields; with practical experience developing software and an intimate knowledge of the challenges involved, but like most developers these people are driven by a passion for their subject, and have strong opinions on the tools and techniques they consider important. The survey has to be impartial so we try to ensure that all the experts are equally unhappy, for balance.

Software development is a global industry these days, so the survey has to reach a global audience. Once the questions have been written, discussed, dismantled, and rebuilt to a mutually-acceptable level of dissatisfaction, then they have to be translated into almost a dozen languages, always ensuring that the clarity of the original remains intact.

Designing developer surveys once more

And then it is done. Perhaps not quite a joy forever, as Keats would have it, but certainly a thing of utility. Questions laid out, check boxes ready to be checked, radio buttons ready to be… radioed(?). Everything waiting for the thousands of developers such as you (what do you mean you haven’t done it yet? Get over there now, this minute). They are drawn by the desire to contribute to the project, or get access to some of the results, or win a prize in the draw, or just know that their opinion matters to the companies and organisations which will be referring to the data over the next six months before the whole process kicks off again.

Shakespeare’s Hamlet asserted that “To be, or not to be” was “the question”, but in these days of Continuous Delivery the questions will never end, and we have turn to a pair of Hamlet’s school friends (Rosencrantz and Guildenstern, with the help of Tom Stoppard) to see where that might lead us: