Related Experiment Video
Updated: May 29, 2026
08:33
A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
Published on: September 4, 2019
How to write high-quality questions for a high-stakes examination
1MRCP(UK) Central Office, Royal College of Physicians, London. charles.twort@gstt.nhs.uk
Clinical Medicine (London, England)
|September 10, 2011
Summary
Medical examinations like the MRCP(UK) and specialty certificate examinations (SCEs) utilize single-best-answer questions. Rigorous peer review and feedback enhance question quality and writer competence for physician assessment.
Area of Science:
- Medical Education
- Assessment and Evaluation
Background:
- Physician training assessments, including MRCP(UK) and SCEs, rely on single-best-answer questions.
- Extensive question banks have been developed since 1999 by specialists and trainees to cover medical curricula.
Purpose of the Study:
- To describe the process of creating high-quality single-best-answer questions for medical examinations.
- To highlight the methods used to ensure questions effectively assess physician knowledge and problem-solving skills.
Main Methods:
- Utilizing question-writing workshops with guidance on designing discriminating questions.
- Implementing multi-stage academic scrutiny, including face-to-face peer review.
- Analyzing individual question performance data.
Main Results:
- A robust system for question development and refinement has been established.
- Feedback mechanisms support the development of question-writing competence among contributors.
- Analysis of question performance offers insights into effective question design.
Conclusions:
- The established processes ensure the quality and effectiveness of questions used in physician training examinations.
- Continuous improvement in question design is facilitated by peer review and performance analysis.
- These methods contribute to accurate assessment of physicians' knowledge and problem-solving abilities.
Related Concept Videos
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:19Quality Assurance
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
02:16Surveys
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
01:19Critical Thinking
Critical thinking involves reflective and productive thinking and the evaluation of evidence. Critical thinkers seek to understand the deeper meaning of ideas, question assumptions, and make independent decisions about what to believe or do. Scientists, for instance, are often critical thinkers. Critical thinking also requires humility about what we know and don't know and the motivation to look beyond the obvious. It is essential for effective problem-solving.
Colleges and universities are...
Colleges and universities are...
01:37Social Loafing
Another way in which a group presence can affect performance is social loafing—the exertion of less effort by a person working together with a group. Social loafing occurs when our individual performance cannot be evaluated separately from the group. Thus, group performance declines on easy tasks (Karau & Williams, 1993). Essentially individual group members loaf and let other group members pick up the slack. Because each individual’s efforts cannot be evaluated, individuals become less...
01:05Quality Control
Quality control is one of the three cyclical quality assurance activities that help keep a system under statistical control. Typical quality control activities include creating quality control charts, conducting proficiency testing, and documenting and archiving results.
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...
Quality control helps track data, visualize trends, and identify variations, making it easier to detect deviations that may affect the accuracy of an analysis. One way to do this is by generating a quality control chart, which...