How Multiple-Choice Questions Work in Surveys

Many surveys that SurveyCompare hosts comprise multiple-choice questions. While these admittedly do narrow down the answers, they speed up the process and simplify analysis later. In this article we tackle a few old chestnuts and try our best to answer them.

• Are surveys accurate
• Do they put words in peoples' mouths
• Why the format of the questions
• How best to answer them
• Are the results reliable
• Should you crook the answers

The first multiple-choice questions tested the intelligence of First World War American recruits almost a hundred years ago. These days, they feature in many more applications, including testing knowledge, gauging opinions and conducting the South African census when it comes around. Ticking a box - or overlooking it - is a binary decision. This makes multiple choices perfect for computer analysis and quick results.

Are Multiple-Choice Surveys Accurate?

The characteristics I mentioned are ideal for market research and opinion surveys such as we feature on SurveyCompare. Opinion seekers love them because the answers are unambiguous and simple to analyse. People taking our surveys welcome them, because the job is over quickly.

Could this be a marriage made in heaven? Multiple-choice questions and surveys seem ideal stable-mates. Scientists who use the system because it is quick and easy, have pointed out that the quality of results does not only depend on the answers given. It depends equally on how well the researcher posed the questions.

Are the Questions Really Questions?

Actually, they are not, in the truest sense at least. Here it is important to discriminate between open and closed questions. For example, if your partner asks "do you love me" there are two possibilities. On the other hand if you are in Sandton and your partner asks "where would you like to eat out tonight" you have a far wider choice.

Multiple-choice surveys restrict the range of answers, because it is easier to analyse five as opposed to a hundred of them. Designers also place them in a narrow context by presenting a stem and then a set of options. For example, in an IQ Test they might ask:

If a=1 and b=2. What is a + b?

1. 12
2. 3
3. 4
4. 10
5. 8

That one is easy. I should think most people know the answer is three. If you don't, this could be because you are not numerate, or your brain does not do numbers. You need a different question. This should be culture fair, such as:

What do South African people call a marriage gift?

1. Lobola
2. Dowry
3. Lobolo
4. Mahadi
5. All of the above
6. A, C & D only

Testers like a mix of simple and more complex questions such as these examples illustrate. They are testing whether the respondent is considering all the options, or just ticking the first answer that comes to mind. In case you're curious, F is the right answer.

How to Answer Multiple Choice Surveys

It is pointless to take a light-hearted approach to a survey. After all, you are receiving money for it and you must put your best foot forward. I have answered so many multiple-choice questions that I have lost count. What I do remember is my method of how to cope with them. I am going to share my secret with you now.

Imagine the question asks "which of the following supermarkets uses the slogan "just up your street". Before you confuse yourself with all the options, decide first what you believe is the most likely one. By now, you may already have something in mind.

Consider the following options next:

1. Woolworths
2. The Orange Farm spaza shops
3. Checkers
4. A & B
5. All of the above

Answering the question becomes far easier after you have thought about things first. Orange Farm spaza shops are in serious competition are unlikely to share slogans. The same is likely when it comes to Woolworths and Checkers. Remember the jingle "Check check Checkers, just up your street". The correct answer is C.

How Long Should the Survey Take?

Most surveys are online on the Internet. The people running them do not want their servers held up by countless people taking all day to answer. Assuming the time allowed is one hour and there are thirty questions, this means an average two minutes each. When I can, I halve that to make sure I finish the whole survey.

One way to be sure of doing this, is to skip on to the next question if the current one is holding you up, and get back to it later when your subconscious has worked it out. Check the instructions carefully before you start. If the system does not allow backtracking, abandon the question just before you reach the allotted time you set. It is better to respond to 29 questions than to waste half the time on one.

Should You Crook the Answers?

The simple answer to this question is no. In fact, SurveyCompare believes that you should log off from our website if this is what you have in mind. Survey designers set trick questions that easily detect if a respondent is lying. If they do:

• They discard your survey form
• They may pay you
• The opportunity is gone forever




In Conclusion

There is more to doing surveys than ticking boxes. The people doing the survey need to know that they can trust you. They will reward you if you do so. In a small way you will have helped move the quality of South African merchandising forward. This in itself makes participating in a survey worthwhile. Through SurveyCompare you get paid for this too.