Designing effective multiple choice questions

A multiple choice question (MCQ) is composed of two parts:

a stem that identifies the question or problem, and
a set of alternatives or possible answers that contain:

a key that is the best answer to the question, and
a number of distractors that are plausible but incorrect answers to the question. (Centre for Teaching Excellence, University of Waterloo, 2012)

Research shows that MCQs are an appropriate competency test for measuring knowledge and comprehension, and can be designed to measure application and analysis. Well-constructed MCQs are also capable of testing the higher levels of cognitive reasoning and can efficiently discriminate between high- and low-achieving students (Abdulghani et al., 2015). However, MCQs are only effective if they are well-designed. Some design principles for MCQs are outlined below.

Designing stems

Express the full problem in the stem

When creating the item, ask yourself if the students would be able to answer the question without looking at the options. This makes the purpose of the question clear. (Centre for Teaching Excellence, University of Waterloo, 2012)

Put all relevant material in the stem

Do not repeat in each of the alternatives information that can be included in the stem. This makes options easier to read and understand, and makes it easier for students to answer the question quickly. (Centre for Teaching Excellence, University of Waterloo, 2012)

Eliminate excessive wording and irrelevant information from the stem

Irrelevant information in the stem confuses students and leads them to waste time. (Centre for Teaching Excellence, University of Waterloo, 2012)

Designing alternative answers

Limit the number of alternatives

Use between three and five alternatives per question. Research shows that three-choice items are about as effective as four or five-choice items, mainly because it is difficult to come up with plausible distractors. (Centre for Teaching Excellence, University of Waterloo, 2012)

All alternatives should be plausible

The function of the incorrect alternatives is to serve as distractors, which should be selected by students who did not achieve the learning outcome but ignored by students who did achieve the learning outcome. (Brame, 2012)

Alternatives should be stated clearly and concisely

Items that are excessively wordy assess students’ reading ability rather than their attainment of the learning objective. (Brame, 2012)

Alternatives should be free from clues about which response is correct

This means that alternatives should:

• have grammar consistent with the stem,

• be parallel in form,

• be similar in length, and

• use similar language. (Brame, 2012)

The alternatives “all of the above” and “none of the above” should not be used

When “all of the above” is used as an answer, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s). When “none of the above” is used as an alternative, test-takers who can eliminate a single option can thereby eliminate a second option. In either case, students can use partial knowledge to arrive at a correct answer. (Brame, 2012)

Attribution

Text extracted and adapted from:

Abdulghani, H. M., Ahmad, F., Irshad, M., Khalil, M. S., Al-Shaikh, G. K., Syed, S., Aldrees, A. A., Alrowais, N., & Haque, S. (2015). Faculty development programs improve the quality of Multiple Choice Questions items’ writing. Scientific Reports, 5(1), 9556. https://doi.org/10.1038/srep09556 Licensed under a CC-BY 4.0 licence.

Brame, C. (2012, December 7). Writing good multiple choice test questions. Center for Teaching, Vanderbilt University. https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/ Licensed under a CC-BY-NC licence.

Centre for Teaching Excellence, University of Waterloo. (2012, November 6). Designing multiple-choice questions. https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/developing-assignments/assignment-design/designing-multiple-choice-questions Licensed under a CC-BY-NC licence.

Image: “life is like a box of questions” by danna & curious tangles, licensed under a CC-BY-NC-ND 2.0 licence.