Do physicians know when their diagnoses are correct? Implications for decision support and error reduction

Prior research has shown that diagnostic accuracy varies with clinical experience, but the relationship between confidence and correctness remains unclear. It was already known that medical trainees often overestimate their diagnostic abilities. No prior work had resolved how this dynamic shifts with experience. This gap motivated the current investigation into how confidence aligns with correctness across different career stages. Established knowledge includes the role of differential diagnosis in clinical reasoning. However, uncertainty remains about whether confidence reliably reflects diagnostic accuracy. That uncertainty drove the need to examine alignment across three groups: students, residents, and faculty. No prior work had resolved whether experience reduces overconfidence in diagnostic settings.

The aim of this study was to assess how well physicians' diagnostic confidence matches the actual correctness of their diagnoses. The specific problem addressed is whether confidence is a reliable indicator of diagnostic accuracy. This question is important for designing decision support systems that reduce errors. The motivation stems from the need to improve clinical decision-making tools. The study sought to compare alignment across three experience levels: students, residents, and faculty. The research focused on diagnostically challenging cases to test real-world relevance. No prior work had resolved whether experience correlates with better calibration of confidence. This study aimed to clarify how confidence and correctness interact in diagnostic reasoning.

The study used a prospective, counterbalanced experimental design. Three groups were tested: senior medical students, residents, and faculty. Each participant received nine detailed case synopses with definitive diagnoses. Participants generated differential diagnoses and rated their confidence. Correctness was defined by whether the true diagnosis was listed. Confidence was measured by likelihood of seeking assistance. Data collection occurred in controlled laboratory settings at three academic centers. The method focused on diagnostically challenging cases to test real-world relevance.

Overall confidence and correctness were only mildly aligned (kappa=0.314). Faculty showed moderate alignment (kappa=0.285), residents lower (0.227), and students higher (0.349). Residents were overconfident in 41% of mismatched cases. Faculty were overconfident in 36% of such cases. Students were overconfident in 25% of mismatched cases. No group showed strong correlation between confidence and correctness. The results suggest that experience does not eliminate diagnostic overconfidence. These findings challenge assumptions about the reliability of self-assessment in clinical settings. The data indicate that confidence is not a reliable indicator of correctness.

The authors state that even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. They propose that medical decision support systems cannot rely solely on clinicians' perceptions of their need for assistance. The findings suggest that confidence is not a reliable indicator of diagnostic accuracy. The authors suggest that interventions to reduce errors must account for this misalignment. They note that overconfidence is common across all experience levels. The authors propose that training programs should address calibration of confidence. They state that current decision support tools may be insufficient if they depend on clinician self-assessment. The authors conclude that further research is needed to develop better alignment between confidence and correctness.