Options in a multiple-choice English test: Quality over quantity

Abstract

Multiple choice tests are useful for testing a large group of students since scoring is objective and fast. For this reason, they are widely used in various standardized tests. However, it takes time to develop good multiple choice items since it is quite challenging to generate distracters that are attractive and have high discrimination power. The question is how many distracters are optimal in realizing the goal of multiple choice tests. Views on this matter in the literature are mixed. The present study compared the effects of three option formats (3, 4, and 5 options) in terms of item discrimination power. Contrary to the popular beliefs, the study found that the discriminatory power decreased as the number of options increased. In other words, the items with 5 options were found to have lowest item discrimination. Considering that the Korean SAT adopts the 5-option format, the implication of the findings is of great significance to language testing practitioners and researchers alike. It can be inferred from the finding that the quality of each option is much more important than the number of options in designing test items.