Day 1 | 09:00 - 10:00 | Keynote address | Gordon Willis

Questionnaire Development, Design, and Testing: Where in the World Are We?

Power Point presentation

Gordon's keynote address on day one set the scene for the conference. He was a very entertaining presenter and massively respected in the community. Ashleigh was very lucky to have attended the short course he delivered on the Wednesday.

Gordon focused on three main questions in the course of his presentation:

1. Where are we today?
2. What does this mean? And, why does it matter?
3. Where are we going?

Where are we today?

Gordon revisited the 4-stage model of cognition, characterising them as the "Four Horsemen of the Questionnaire Design Apocolypse":

• Comprehend
• Retrieve
• Decision/Judgement
• Response matching

Gordon proposed that modern questionnaire development has given rise to a fifth horseman:

• Structural/Logical

Spoke of methods used to develop questions, using a model exploring relationships/interactions between Investigator/Designer, Survey Question, Respondent (sometimes via an interviewer)  all set within a cultural and environmental context.

Slides 12 - 22 in the presentation explore these elements and where the various evaluative methods focus.

What does this mean? And, why does it matter?

Gordon contended that, despite the move to big data, emergence of non-probability sampling etc the questionnaire still really matters. But he did note that we will need to evolve, using the analogy of the emergence of the motor car. We need to realise the car is coming and that people will not be satisfied with better ways to drive the horse and cart (i.e. people will not be satisfied by a better whip when they could have a car). 

Gordon suggested the at meeting the challenges of questionnaire design in the modern environment requires us to "Return to motivation" to encourage response and engagement with our surveys. 

• Increase attractiveness to "digital native multi taskers" (the types of people who surf their phones while streaming Netflix)
• Switch to devices better matching modern response
• Go to non-questionnaire data

Where are we going?

Gordon concluded by talking about "Where he sees the discipline going" or at least where he would like it to be going:

• Better describe what we do (“data scientists”)
• Disseminate what we find (eg Qbank)
• Communicate what we think works
• Criteria for declaring victory and going home (reaching saturation)

Humorous model of context being so important as to what is the right way to do things. The "Toilet Paper Model" (why I choose to hang toilet paper the "Wrong Way" - slide 46)