Physicians' diagnostic accuracy, confidence, and resource requests: a vignette study.

Physicians routinely assess their own diagnostic accuracy, but the relationship between their confidence levels and actual correctness remains poorly understood. While prior research has shown that diagnostic accuracy varies with case complexity, the interplay between confidence and accuracy is less clear. This gap motivated a closer examination of how diagnostic calibration—the match between accuracy and confidence—shifts as cases become more complex. Existing studies have established that physicians face challenges in diagnosing difficult cases. However, no prior work had resolved how confidence levels change with case difficulty or how this affects resource requests. The need for clarity arises from the potential impact on patient care when physicians overestimate their accuracy. This paper's contribution lies in its structured analysis of diagnostic calibration across evolving case phases and difficulty levels. The study addresses a specific gap in understanding how physicians adjust their confidence and diagnostic strategies as cases progress. It also explores the role of resource requests in response to diagnostic uncertainty.

The study aimed to evaluate how physicians' diagnostic calibration changes as cases evolve and as diagnostic difficulty increases. Researchers sought to determine whether confidence levels align with actual diagnostic accuracy across different stages of the diagnostic process. The specific problem addressed is the potential mismatch between physicians' confidence and their actual diagnostic performance. The motivation stems from the need to understand how diagnostic calibration affects clinical decision-making. The study also aimed to examine how physicians respond to uncertainty by requesting additional resources. By analyzing how confidence and accuracy interact, the research sought to identify patterns in physician behavior. The goal was to determine if confidence is a reliable indicator of diagnostic accuracy. The study's design allowed for a detailed look at how physicians adjust their approaches as cases become more complex.

The study involved general internists recruited from an online physician community. Participants were asked to diagnose four validated clinical case vignettes of varying difficulty. Each case was presented in four sequential phases simulating the diagnostic process: history, physical examination, general diagnostic testing, and definitive diagnostic testing. After each phase, physicians recorded up to three differential diagnoses and their confidence levels. Before seeing definitive data, participants were asked what additional resources they would need, such as tests, consultations, or reference materials. Diagnostic accuracy was scored as correct or incorrect. Confidence levels were measured on a 0-10 scale. The study also tracked whether physicians requested additional resources. The design allowed for a detailed analysis of how confidence and accuracy interacted across different case phases and difficulty levels.

Physicians correctly diagnosed 55.3% of easier cases and 5.8% of more difficult ones (P < .001). Despite the large difference in accuracy, confidence levels were only slightly lower for difficult cases (6.4 vs 7.2 out of 10). This suggests a mismatch between accuracy and confidence. Diagnostic calibration was worse for more difficult cases (P < .001), indicating overconfidence. Physicians with higher confidence were less likely to request additional diagnostic tests (P = .01). More difficult cases led to more requests for reference materials (P = .01). The results show that confidence was relatively insensitive to case difficulty. Physicians tended to overestimate their accuracy when cases were complex. These findings highlight a potential risk in clinical decision-making when confidence does not reflect actual diagnostic performance.

The study suggests that physicians' confidence may not accurately reflect their diagnostic accuracy, especially in difficult cases. The authors found that diagnostic calibration worsened as case difficulty increased (P < .001). This overconfidence could lead to missed opportunities for reevaluation. The mismatch between confidence and accuracy was more pronounced in complex cases. Physicians with higher confidence were less likely to request additional diagnostic tests (P = .01). The results indicate that confidence levels are relatively insensitive to case difficulty. The authors propose that this pattern could hinder physicians from reexamining difficult cases where their initial diagnosis may be incorrect. The findings suggest that confidence is not a reliable indicator of diagnostic accuracy in complex clinical scenarios.