Related Experiment Video
Updated: Jan 20, 2026
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Incorporating Physician Input Into a Maintenance of Certification Examination: A Content Validity Tool
Paul A Poniatowski1, Jeremy W Dugosh, Rebecca A Baranowski
1P.A. Poniatowski is former vice president, Test Development, American Board of Internal Medicine, Philadelphia, Pennsylvania. J.W. Dugosh is vice president, Test Development, American Board of Internal Medicine, Philadelphia, Pennsylvania. R.A. Baranowski is director, Test Development Operations, American Board of Internal Medicine, Philadelphia, Pennsylvania. G. Arnold is former senior statistician, Research and Innovations, American Board of Internal Medicine, Philadelphia, Pennsylvania. R.S. Lipner is senior vice president, Assessment and Research, American Board of Internal Medicine, Philadelphia, Pennsylvania. G.W. Dec Jr is chief emeritus, Cardiology Division, Massachusetts General Hospital, Boston, Massachusetts, and Roman W. DeSanctis Professor of Medicine, Harvard Medical School, Boston, Massachusetts. M.M. Green is senior associate dean for Medical Education, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Purpose:
As part of the American Board of Internal Medicine's (ABIM's) continuing effort to update its Maintenance of Certification (MOC) program, a content validity tool was used to conduct structured reviews of MOC exam blueprints (i.e., test specification tables) by the physician community. Results from the Cardiovascular Disease MOC blueprint review are presented to illustrate the process ABIM conducted for several internal medicine disciplines.
Method:
Ratings of topic frequency and importance were collected from cardiologists in 2016 using a three-point scale (low, medium, high). The web-based survey instrument presented 188 blueprint topic descriptions, each combined with five patient-related tasks (e.g., diagnosis, treatment). Descriptive statistics and chi-square analysis were employed.
Results:
Responses from 441 review participants were analyzed. Frequency and importance ratings were aggregated as a composite statistic representing clinical relevance, and exam assembly criteria were modified to select questions, or items, addressing clinically relevant content only. Specifically, ≥ 88% of exam items now address high-importance topics, including ≤ 15% on topics that are also low frequency; and ≤ 12% of exam items now address medium-importance topics, including ≤ 3% on topics that are also low frequency. The updated blueprint has been published for test takers and provides enhanced information on content that would and would not be tested in subsequent examinations. It is linked to more detailed feedback that examinees receive on items answered incorrectly.
Conclusions:
The blueprint review garnered valuable feedback from the physician community and provided new evidence for the content validity of the Cardiovascular Disease MOC exam.
Related Concept Videos
10:39Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
07:44Ophthalmoscopic Examination
The simplest ophthalmoscopes consist of an aperture to look through, a diopter indicator, and a disc for selecting lenses. The ophthalmoscope is primarily used to examine the fundus, or the inner wall of the posterior eye, which consists of the choroid, retina, fovea, macula, optic disc, and retinal vessels (Figure 1). The spherical eyeball...
10:54C. elegans Maintenance
In this video you will learn the basic housing and feeding requirements of C. elegans, how to correctly handle and manipulate worms using a worm pick and how to freeze and recover...
06:40The Dyspepsia Educational Tool As a Novel Aid in Dyspepsia Management
12:35Anesthesia Induction and Maintenance
The Guide for the Care and Use of Laboratory Animals ("The Guide") states that pain assessment and alleviation are integral components of the veterinary care of laboratory animals.1 The definition of anesthesia is the loss of feeling or sensation. It is a dynamic event involving changes in anesthetic depth with respect to an animal's metabolism, surgical stimulation, or variations...
14:18The 4 M’s Framework: Case History and Physical Examination of Older Adults
To meet the needs of older adults, all health professionals should be well acquainted with the history and physical examination considerations unique to this population. Physical examination plays an important role in the older patient to detect physiologic changes of aging, risk factors, and signs of pathology. While most of the general principles of the standard examination for adults apply to older patients,...