97 Things Every UX Practitioner Should Know

Chapter 87. If Designing Survey Questions Were Easy, There’d Be No Garbage Data

Annie Persson

Surveys are a fast, efficient, and popular way to complement and validate your qualitative data. They’re one of the most accessible quantitative methodologies you can employ, but there’s more to a survey than just quickly writing questions. How you design questions can greatly impact the validity of your research.

I designed a survey to understand how my employer used our company knowledge base and community forum. I thought I knew what I was doing by designing the question closed-ended with a “type in your answer” response option. But then I got a whiff of my own garbage data.

My question (garbage in): Do you refer instructors or students to the knowledge base and community forum for getting help with their answers?

My responses (garbage out): Yes, No, Sometimes

Part of designing questions for a survey is deciding whether to use open-ended questions (respondents answer in their own words) or closed-ended questions (respondents select from a set of choices). If your goal is to achieve statistical significance, capture data from a large target population using closed-ended questions.

If I could tell my younger self how to rewrite that question today, I would construct it as closed-ended, group it into separate questions, and eliminate the leading verbiage:

Do you refer instructors ...