Evaluating the Application of an EMS Clinical Judgment Theoretical Framework

Clinical judgment (CJ) encompasses clinical reasoning (process of evaluating a problem) and clinical decision-making (choice made). A theoretical model to better define emergency medical services (EMS) CJ has been developed but its use has not been evaluated in EMS training and assessments. Our objective was to evaluate the performance of this EMS CJ model to assess clinical reasoning and decision-making in a simulated environment.

In this evaluation, EMS clinician teams (2-3 members) were directed to care for a simulated older adult patient in their home following a fall. Simulations were video recorded, clinician team actions coded, and evaluated for whether proper CJ reasoning and decisions were made. We evaluated CJ in two ways: 1) EMS medical directors' (MD) determination of whether the CJ questions were addressed (MD score) and 2) objective rubric evaluation of CJ questions using the EMS CJ model focused on recognition of appropriate cues, performance of actions, and revaluation after action (rubric score). The CJ questions addressed in this simulation included: 1) Is the patient stable/unstable?, 2) Are interventions necessary before movement?, 3) How should the patient be transferred from the floor?, and 4) Does the cause of the fall require hospital evaluation? Descriptive statistics were calculated, and concordance between the two assessments was evaluated (mean, 95% CI). Percent concordance was calculated with a validity threshold set at 70%.

Four EMS MDs reviewed 20 videos addressing 80 clinical judgment decisions. Overall concordance between MD score and rubric score for CJ decisions was above the threshold at 88.1% (85.0, 91.2). Concordance between MD score and rubric score for each CJ decision was 92.0% (87.3, 96.7) for question 1, 79.9% (71.5, 88.3) for question 2, 95.0% (90.4, 99.6) for question 3, and 85.4% (79.5, 91.2) for question 4.

An objective evaluation of CJ decisions using a rubric derived from an EMS CJ theoretical framework demonstrated high concordance to subjective evaluations of CJ made by EMS MDs. This approach may allow for reproducible and objective CJ evaluations that could be used for competency assessment in EMS.