Selecting cases for feedback to pre-hospital clinicians - a pilot study

Currently, there is no standard process for hospitals to provide feedback to pre-hospital clinicians about their diagnostic accuracy. While pre-hospital clinicians make critical early assessments, these decisions are rarely reviewed in a structured way. It was already known that pre-hospital care involves high-stakes decision-making, but formal feedback mechanisms remain limited. This gap motivated the need to explore whether selective case identification could serve as a feedback tool. No prior work had resolved how to systematically extract and evaluate cases for diagnostic discrepancies. Prior research has shown that diagnostic accuracy in pre-hospital settings can vary widely. However, the extent of these discrepancies and their clinical significance remained unclear. This paper adds a new approach to identifying cases where feedback could be most useful.

The goal was to test a method for identifying cases where pre-hospital and in-hospital diagnoses differ in a clinically meaningful way. This approach could help inform feedback systems for pre-hospital clinicians. The study aimed to determine whether such discrepancies could be reliably detected and quantified. The motivation stemmed from the need to improve diagnostic accuracy in pre-hospital care. By identifying these cases, the study sought to lay the groundwork for targeted feedback. The researchers propose that selective feedback could lead to improved clinical decision-making. No prior work had demonstrated a reliable way to extract and assess these diagnostic differences. This study aimed to fill that gap through a structured, multi-step process.

The study used a single-centre, retrospective design involving randomly selected cases from hospital discharge records. Informatics tools were used to extract final diagnoses and patient data. Explicit chart reviews were conducted to gather additional clinical details. Two blinded groups of clinicians assessed the data independently. They evaluated whether pre-hospital and in-hospital diagnoses showed clinically significant differences. Inter-rater reliability was measured using a kappa score. The study combined automated data extraction with clinical consensus to identify discrepancies. This mixed-methods approach allowed for both quantitative and qualitative assessments.

Out of 353 cases, 32 (9.1%) showed clinically significant differences in diagnosis between pre-hospital and in-hospital assessments. The majority of patients had high triage scores (Australasian Triage Scale categories 1–3). In-hospital mortality was 32.9% among these patients. The inter-rater reliability between clinician groups was moderate (kappa score 0.6). The confidence interval for the discrepancy rate was 6.1–12.1%. These discrepancies were most common in trauma and acute myocardial infarction cases. The study found that diagnostic differences were not uniformly distributed across patient types. The results suggest that a small but meaningful proportion of cases could benefit from targeted feedback.

The study found that a small but clinically relevant proportion of cases had discrepancies between pre-hospital and in-hospital diagnoses. These findings suggest that selective case identification could support feedback mechanisms for pre-hospital clinicians. The researchers propose that such feedback could improve clinical decision-making. The study's approach combined informatics and clinician consensus effectively. The results do not suggest that all cases require feedback, but rather that a subset could be prioritized. The authors state that feedback systems should focus on cases with the greatest potential for improvement. No essential role of any single diagnostic category was claimed. The study supports the idea that targeted feedback can be a practical tool for pre-hospital care.