Evaluating Acute Stroke Diagnosis Using Simulation Scenarios

This study sought to explore the clinical and process factors that, alone or in combination, influence the accuracy of stroke diagnosis in the emergency department (ED) by applying a novel design of experiments methodology.

We used design of experiments, a branch of applied statistics, to create a screening experiment comprised of patient simulation scenarios in which purposeful changes are made to prespecified factors thought to potentially influence the outcome of interest. We used 4 base case scenarios (typical and atypical acute ischemic stroke, intracerebral hemorrhage, and complex migraine [a stroke mimic]) and 12 prespecified clinical factors thought to influence stroke diagnosis (eg, patient age, history of seizure, and interruption occurring during patient assessment that the physician must attend to [distraction]) based on literature review and expert opinion. Experimental runs were designed using a modified fractional factorial design approach. Physicians, including postgraduate trainees from 3 health systems, were invited to participate. After each run, participants were asked to provide a diagnosis and their confidence in that diagnosis; both inperson and virtual assessments were conducted. We used multivariate regression to explore factor(s) influencing physician confidence in stroke diagnosis. Confidence was signed, with positive confidence corresponding to a stroke diagnosis and negative confidence to a nonstroke diagnosis, allowing confidence levels from the regression model to be translated to misdiagnosis.

A total of 27 physicians (12 neurology and 15 emergency medicine) completed 100 experimental runs. The typical ischemic stroke base case presentation was accurately diagnosed in every run, whereas the other 3 base cases were less accurately diagnosed. Stroke overdiagnosis occurred in the complex migraine case (83% false positive) and stroke underdiagnosis in the intracerebral hemorrhage case (36% false negative). Distraction during patient evaluation and the availability of a witness from whom relevant information could be obtained exhibited significant, independent effects on diagnostic confidence. Distractions of the physician had an even stronger influence on stroke diagnostic confidence when no witness was present.

Applying the design of experiments methods to simulated scenarios, we found that distraction and presence of a witness significantly influenced diagnostic confidence and thus, stroke diagnostic accuracy. These findings should inform future studies to understand the underlying mechanisms of ED misdiagnosis and in the design of future interventions to improve stroke diagnostic formulation.