A Target Population Derived Method for Developing a Competency Standard in Radiograph Interpretation.

Area of Science:

• Medical Education
• Radiology
• Diagnostic Skills

Background:

• Current methods for assessing radiograph interpretation competency are often subjective and lack connection to clinical populations.
• Existing digital platforms lack data-driven competency standards, failing to assure adequate physician mastery for clinical practice.

Purpose of the Study:

• To adapt the Ebel Method for setting a clinically relevant competency standard for radiograph interpretation.
• To establish a data-driven model for deriving competency thresholds in visual diagnosis tasks.

Main Methods:

• Adapted the Ebel Method using a purposeful sample of pediatric musculoskeletal extremity radiographs.
• Utilized item response theory to categorize radiograph difficulty and expert ratings for clinical significance.
• Developed a three-by-three matrix and involved multidisciplinary stakeholders to determine acceptable accuracy levels.

Main Results:

• 244 emergency physicians interpreted 1,835 radiographs, with difficulty varying significantly by body region.
• A performance-based competency standard of 85.5% accuracy was derived, with acceptable scores ranging from 76% to 95%.
• Potential diagnostic errors at 85.5% accuracy were predominantly in low-significance radiographs (9.8%).

Conclusions:

• The study presents a novel, evidence-based method for setting radiograph interpretation competency standards.
• This approach provides a model for deriving competency thresholds in other visual diagnosis clinical tasks.
• Empirically driven standards enhance the reliability of assessing physician competence for patient safety.