Development and Validation of a Formative Assessment Tool for Nephrology Fellows' Clinical Reasoning

Clinical reasoning is a critical skill for medical trainees, yet deficits are common. Prior research has shown that graduate medical learners, including nephrology fellows, often struggle with effective clinical reasoning. While existing studies have explored general diagnostic errors, few have focused on structured assessment tools specific to nephrology. This gap motivated the development of a targeted formative assessment. No prior work had resolved how best to measure clinical reasoning in nephrology fellows using simulated patient encounters. Prior approaches lacked validated thresholds for remediation. This study addresses these limitations by introducing a new instrument. The tool aims to provide actionable data for coaching. It is designed to work within the unique context of nephrology training. The study contributes a novel method for evaluating clinical reasoning in this specialty.

The study aimed to develop and validate a formative assessment tool for nephrology fellows' clinical reasoning. The specific problem addressed is the lack of reliable instruments to evaluate clinical reasoning in nephrology trainees. The motivation stems from the high prevalence of diagnostic errors in this field. The authors sought to create a practical tool for formative feedback. The tool was designed to measure three key domains of clinical reasoning. The study tested the tool's reliability and established performance thresholds. It aimed to determine if the instrument could identify fellows needing remedial coaching. The ultimate goal was to support targeted instruction in clinical reasoning.

The researchers designed an assessment instrument based on a simulated acute kidney injury consult. The instrument evaluated three domains: problem representation, differential diagnosis, and diagnostic plan. Expert raters in nephrology education and clinical reasoning developed the scoring criteria. A pilot cohort of first-year nephrology fellows tested the tool's reliability. Inter-rater reliability was measured using the intraclass correlation coefficient. The Hofstee method was used to establish performance thresholds in a larger cohort. Six raters scored the assessments in the main study. The study included 15 fellows in the pilot and 61 in the main cohort.

The instrument demonstrated strong inter-rater reliability in the pilot cohort. Problem representation had an ICC of 0.90, differential diagnosis had 0.70, and diagnostic plan had 0.50. Passing thresholds were set at 59%, 57%, and 62% for the three domains. Fifty-nine percent of fellows met the threshold for remedial coaching in at least one area. The study provided validity evidence for the simulated AKI consult. The results suggest the tool can identify fellows needing targeted instruction. The thresholds offer a basis for formative feedback. The findings support the use of this instrument in nephrology training programs.

The authors propose that the assessment instrument is a valid tool for evaluating clinical reasoning in nephrology fellows. The study supports the use of the simulated AKI consult as a formative assessment method. The results suggest that most fellows require coaching in at least one domain. The thresholds provide a clear benchmark for remediation. The instrument's reliability and validity support its use in training programs. The findings highlight the need for targeted instruction in clinical reasoning. The study does not claim the tool is the only method for assessment. The authors state that the instrument can guide formative feedback and instruction.