Development and Pilot Evaluation of the ECALM Program: A Bedside Clinical Assessment Tool for First-Year Residents in...

To describe the development and pilot evaluation of the ECALM program ("bedside clinical evaluation", in French), a formative bedside assessment framework designed for first-year residents during emergency department (ED) rotations.

The ECALM program was developed prospectively in three stages, in accordance with Stufflebeam's "context, input, process, product" framework: (i) a baseline needs assessment survey, (ii) the development of standardized clinical scenarios via a structured expert consensus process, and (iii) a single-center pilot implementation. Six core clinical themes were selected (chest pain, abdominal pain, limb trauma, syncope, fever, and confusion), on the basis of epidemiological data from French EDs. A pilot study involving 12 first-year residents was conducted at Amiens University Medical Center's ED (Amiens, France). Each resident was evaluated with a single scenario, and each session included a structured briefing, passive bedside observation by paired, independent, senior evaluators, and a formative debriefing. Psychometric evaluation of the ECALM tool was based on inter-rater reliability, using the intraclass correlation coefficient (ICC(2,1)) calculated using a two-way random-effects model for absolute agreement based on a single measurement, the weighted kappa, and a Bland-Altman analysis of inter-observer differences.

In the needs assessment (physician response rate: 27.1%), 97% of the participating senior practitioners reported that residents were insufficiently prepared for independent clinical tasks. During the pilot phase, the mean ± standard deviation total duration of an ECALM session was 43 ± 12 minutes. A preliminary analysis of the scores' consistency yielded an ICC(2,1) of 0.76, indicating good inter-rater agreement between independent evaluators. The weighted kappa (0.74) was consistent with the ICC results. A Bland-Altman analysis demonstrated a mean inter-observer difference of only 1.25%, indicating negligible systematic measurement bias between evaluators. Qualitative feedback demonstrated high acceptability: all 12 residents expressed their satisfaction, reported an increased sense of psychological safety, and perceived the session to be a "one-on-one tutorial". Furthermore, ten residents (83.3%) spontaneously reported acquiring new knowledge during the debriefing phase.

The results of this pilot study demonstrated the operational feasibility, high acceptability, and good preliminary inter-rater reliability of the ECALM program for the formative assessment of first-year residents. Although ECALM constitutes a promising, non-punitive approach for encouraging early clinical reasoning and self-assessment, further well-powered, multicenter validation is required to demonstrate the tool's external validity and long-term impact.