Comment by twothamendment
3 years ago
My first programming job was at a survey place. Once some high paid survey writer flew in to get one out in a hurry. I was told to sit with him and build it while he designed it. Ever since then I've though about surveys differently. The level of thought that he put into not only each question, but every word. Considering how one question might influence another would cause a change in order, etc. Anyway, it was fascinating to see his thought process, compared to being handed the finished product like we normally saw.
Yeah, surveys done for actual reasons with skill are amazing.
Most surveys are not done with any such skill - they’re done to get the result desired.
Basically what every student in the social sciences learns.
Every introductory textbook on empirical methods has a chapter on surveys and questionnaire design, and there are also specialized books just on that.
Tech would sometimes really do well in listening to people outside its field.
There are real complexities in designing a survey right. As you say, it's a subject of multiple books and even PhD dissertations.
I worked for 4 years with a team of mainly social scientists (agricultural economists, rural development researchers, etc) for a huge EU project that included plenty of surveys for 6 locations in different countries/languages. The effort to design the necessary surveys right and in different languages/cultures really amazed me. Some of the teams even got (good) papers out of it. There are so many biases that you can have with the wrong questions, groupings or even question orders.
One example I remember is that while in Germany (Altmark region) you could go straight to the people and ask them about their use of policy programme money, what had worked and what hadn't. You could not do the same in Croatia or in Bulgaria (we had study sites in both places), but had to layout questions differently, to get useful answers.