MedEdWorld Updates highlight useful and easily accessible sources of information on key and important topics in medical education contributed by leading experts from around the world.
MedEdWorld
A global online medical education community
U P D AT E O N
MULTIPLE CHOICE QUESTIONS Useful resources on MedEdWorld Dr Catherine Kennedy, AMEE Projects Officer
GLOSSARY
A database of the expanding vocabulary used in medical education
Multiple Choice Questions: ‘Multiple Choice Questions (MCQs) have been a common means of written assessment in medical education for many years. They are suitable for use in areas where objectively correct answers can be ascertained, such as the basic sciences and some clinical scenarios. In general MCQs are designed with a stem – a question or scenario – and a set of responses. MCQs can be implemented in a variety of ways including: • Single best option - a student is expected to select the best response from a range of 4-5 possible options; • Multiple true/false questions – a student has to assess a number of statements relating to the question as either being true or false; • Extended matching questions – students answer a series of questions relating to a scenario by selecting responses from an extended list of approximately 20 options. MCQs have a range of advantages, such as enabling examiners to cover a large degree of content, and are increasingly easy to administer and score using computers. However, designing valid questions and responses is a demanding skill that can be time consuming, and criticism has been made that they do not assess higher-order learning and analytic skills. Further discussion of MCQs and their advantages and disadvantages can be found in our Glossary. The MedEdWorld Glossary can be accessed at www.mededworld.org/Glossary.aspx
WEBINARS
Hour-long interactive presentations with experts in medical education
Developing High-quality Single- Best-Answer MCQs to Assess Application of Knowledge Using Patient Vignettes Presenters: Kathy Holtzman and Dave Swanson Host Institution: National Board of Medical Examiners, USA www.mededworld.org/Webinars/Webinar-Items/DevelopingHigh-quality-Single-Best-Answer-MCQs- t.aspx This webinar focuses on writing high-quality MCQs using patient vignettes that go beyond testing recall of isolated facts to assess application of basic science and clinical knowledge in decision making.
www.mededworld.org
www.amee.org
TEXTBOOKS
Tweed, M., Thompson-Fawcett, M., Schwartz, P., and Wilkinson, T.J. (2013) Determining measures of insight and foresight from responses to multiple choice questions. Medical Teacher, 35:2, pp. 127-133. www.mededworld.org/Publications/Articles/Determining-measures-of-insight-and-foresight-from.aspx Clinical practitioners require insight, the ability to judge their personal capability, and foresight the ability to predict the potential consequences of the actions they suggest.
A selection of texts recommended on MedEdWorld
Case, S.M. and Swanson, D.B. (2002) Constructing Written Test Questions for the Basic and Clinical Sciences (3rd Ed. Rev.) Philadelphia, PA: National Board of Medical Examiners. www.mededworld.org/Publications/Textbooks/Constructing-Written-Test-Questions-For-the-Basic.aspx The manual provides an overview of item formats, concentrating on the traditional one-best-answer and matching formats. It reviews issues related to technical item flaws and item content; statistical indices of item quality after test administration; and standard-setting.
Zoanetti, N., Beaves, M., Griffin, P., and Wallace, E.M. (2013) Fixed or mixed? a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program. BMC Medical Education, March 2013. www.mededworld.org/Publications/Articles/Fixed-or-mixed-a-comparison-of-three,-four-and-mix.aspx This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP).
Haladyna, T.M. and Rodriguez, M.C. (2013) Developing and Validating Test Items. London: Routledge. www.mededworld.org/Publications/Textbooks/Developing-and-Validating-Test-Items.aspx An update and development of ‘Developing and Developing Multiple-Choice Items (3rd Ed.)’ (Haladyna, 2004), this book provides a comprehensive resource for the development and validation of multiplechoice question items and constructed response question formats.
RESOURCES
Osterlind, S.J. (1997) Constructing Test Items: Multiple-Choice, Constructed Response, Performance and Other Formats. Boston: Kluwer Academic Publishing. www.mededworld.org/Publications/Textbooks/Constructing-Test-Items-Multiple-Choice,-Construct.aspx This book addresses major issues in constructing test items by highlighting: the characteristics and functions of test items; editorial guidelines for writing test items in all of the commonly used item formats; methods for determining the quality of test items; procedures for ordering items in a test; item scoring schemes; and computer-generated items.
JOURNAL ARTICLES
Guidelines for writing Multiple Choice Questions (MCQs) Single Best Answer (SBA) & Extended Matching Items (EMI) formats www.mededworld.org/Resources/Resources-Items/Guidelines-for-writing-Multiple-Choice Questions-% 28.aspx Date: 2012 Author: Intercollegiate Speciality Boards This guide provides a range of useful information for the development of MCQs, including Single Best Answer (SBA) and Extended Matching Item (EMI) formats.
A selection of key articles recommended by the MedEdWorld Community
LEARNING TECHNOLOGY
Bandaranayake, R.C. (2008) Setting and maintaining standards in multiple choice examinations: AMEE Guide No. 37. Medical Teacher, 30:9-10, pp. 836-845. www.mededworld.org/Publications/Articles/Setting-and-maintaining-standards-in-multiple-choi.aspx This guide examines the more commonly used methods of standard setting, illustrates the procedure used in each with the help of an example, and discusses the advantages and disadvantages associated with the use of each.
Recommended to support teaching, learning and assessment
Questionbank www.mededworld.org/Learning-Technology/Equipment-Items/QuestionBank-Manage-Exam-Questions. aspx This software offers a toolset for developing, managing and maintaining examination questions, as well as generating detailed analysis on question performance.
Campbell, D.E. (2011) How to write good multiple-choice questions. Journal of Paediatrics and Child Health. 47:6, pp.322-325. www.mededworld.org/Publications/Articles/How-to-write-good-multiple-choice-questions.aspx This paper provides a simple guideline that will assist with the production of high-quality multiple-choice papers.
Questionmark www.mededworld.org/Learning-Technology/Equipment-Items/Questionmark-Enables-measurement-ofknowledge,-ski.aspx Questionmark’s technologies and services enable organizations to measure knowledge, skills, and attitudes securely for certification, regulatory compliance and successful learning outcomes.
McCoubrie, P. (2004) Improving the fairness of multiple-choice questions: a literature review. Medical Teacher, 26:8, pp. 709-712. www.mededworld.org/Publications/Articles/Improving-the-fairness-of-multiple-choice-question.aspx This article reviews the literature on MCQ-testing and the perceptions that they can be ‘unfair’. The article explores these concerns before making a series of practice points for conducting fair and defensible MCQ examinations.
Hot Potatoes www.mededworld.org/Learning-Technology/Equipment-Items/Hot-Potatoes.aspx The Hot Potatoes suite includes six applications, enabling you to create interactive multiple-choice, shortanswer, jumbled-sentence, crossword, matching/ordering and gap-fill exercises for the World Wide Web.
FORUMS
Markert, R.J., Goggans, D.S., and Richardson, W.S. (2012) Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful. Teaching and Learning in Medicine, 24:4, pp. 341-347. www.mededworld.org/Publications/Articles/Local-Development-of-MCQ-Tests-for-Evidence-based.aspx This paper describes a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy’s success.
A space for debating a wide range of topics of current interest to the health professions.
MCQs http://www.mededworld.org/Forums/f129/t4388/MCQs.aspx Post your queries, ideas and suggestions on MCQs here for discussion with colleagues. You can also add your recommendations on topics to a forum for upload to the MedEdWorld site.
Palmer, E.J. and Devitt, P.G. (2007) Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Medical Education, 49:7. www.mededworld.org/Publications/Articles/Assessment-of-higher-order-cognitive-skills-in-und.aspx This study was designed to assess the effectiveness of Modified Essay Questions (MEQs) to measure higherorder cognitive skills in an undergraduate institution.
www.mededworld.org
Including reports, websites, video clips, images and diagrams.
www.amee.org
www.mededworld.org
www.amee.org
MedEdWorld was launched by AMEE to help all with an interest in health professions education learn, connect and debate key issues in medical education. Join the MedEdWorld Community by registering as a user for free or joining as a member at: www.mededworld.org AMEE members have full and free access to MedEdWorld. Simply login with your AMEE username and password.
Copies of this and other MedEdWorld Updates are located in the Resources section of MedEdWorld. This will give you direct access to all the hyperlinks listed in the summary.
www.mededworld.org
Scottish Charity SCO31618
www.amee.org
