Guidelines for developing high quality multiple-choice assessments

Abstract

BACKGROUND: Multiple-choice questions are frequently used in high-stakes assessments across health science disciplines. Many teachers, however, lack the necessary knowledge and training to develop these tests. This paper will discuss test development guidelines to help teachers produce valid and reliable multiple-choice assessments. SUMMARY OF WORK: We reviewed the assessment literature and identified relevant research papers on issues of quality in multiple-choice tests. We then synthesized this research literature and developed a set of sequential quality assurance processes that teachers responsible for test development can use as a guide or an organizational framework for multiple-choice test construction. SUMMARY OF RESULTS: The guidelines for multiple-choice test development can be divided into three categories: (1) pretest planning, (2) test development practices, and (3) posttest review. The relevant research is discussed for each recommendation and practical issues are highlighted. CONCLUSIONS: Quality in educational assessments of health science professionals is an issue receiving increased attention as a result of a greater focus on outcome based assessments in tertiary educational institutions and the increased accountability the public is demanding from institutions that produce health professionals. Hence, clear research-based test development guidelines are required. TAKE-HOME MESSAGES: By following multiple-choice test development guidelines, teachers will be able to develop high quality multiple-choice tests that assess instructional learning objectives and higher cognitive domains.