A Competency Assessment Tool for Liver Contrast-Enhanced Ultrasound Among Sonography Trainees: Development and Multicenter Validation Study

Despite the increasing use of liver contrast-enhanced ultrasound (CEUS), no standardized, organ-specific tool exists for assessing trainees' cognitive competency. Existing instruments lack multidimensional evaluation and external validation, which limit training quality and patient safety.

This study aims to develop and validate the CEUS Competency Assessment for the Liver (CCAL), a tool to standardize the evaluation of theoretical and cognitive skills in liver CEUS.

This retrospective cohort study had four stages: (1) item generation from literature and guidelines, (2) 3 Delphi rounds (12/21, 57.1% female experts; mean age 51.0, SD 7.2 y) and cognitive testing (10 trainees) for content validity (>85% agreement) and clarity, (3) internal validation involving 100 sonography trainees (n=66, 66% assistant professors; n=27, 27% associate professors; n=7, 7% professors; n=70, 70% CEUS-naive) with 1 month or more of training at 4 hospitals (2018-2020), and (4) external validation on 579 sonography trainees (n=299, 51.6% assistant professors; n=183, 31.6% associate professors; n=97, 16.8% professors; n=439, 75.8% CEUS-naive) with 1 or more months of training at 25 multicenter sites (2018-2020). Primary outcomes were reliability (Cronbach α and Guttman split-half) and validity (exploratory factor analysis and item discrimination), while secondary outcomes included item difficulty distribution, pass rates, and between-group differences.

The final CCAL comprised 74 items across 3 domains (demographics: 8, self-evaluation: 13, and objective questions: 53). Internal consistency was excellent (Cronbach α=0.916 and Guttman split-half=0.908). Factor analysis confirmed a 3-factor structure explaining 74.6% of the cumulative variance. Item analysis showed a balanced difficulty distribution (35/53, 66.0% easy; 17/53, 32.1% moderate; 1/53, 1.9% difficult items). Scores varied significantly by CEUS training duration (P<.001); trainees with 4 to 6 months or more than 6 months of training achieved higher self-evaluation (F_2,576=34.751, partial η²=0.108, 95% CI 0.069-0.147) and objective question (F_2,576=34.374, partial η²=0.107, 95% CI 0.068-0.146) scores than those with less than 3 months of training. Prior independent CEUS experience was also associated with higher scores on the self-evaluation (t₅₇₇=6.736, Cohen d=0.561, 95% CI 0.393-0.728; P<.001) and objective question sections (t₅₇₇=3.494, Cohen d=0.291, 95% CI 0.127-0.454; P=.001). Competency gaps were identified across specific competency domains: the pass rate ranged from 32.5% (188/579; image-guided intervention) to 81.3% (471/579; indications and contraindications).

The CCAL is the first liver-specific CEUS competency tool integrating objective knowledge testing with structured self-evaluation. Unlike generic or simulation-based tests, it is organ-specific and validated across a large multicenter cohort, providing a multidimensional framework that identifies specific competency gaps, most notably in advanced image-guided procedures, beyond global pass/fail outcomes. The tool supports competency-based medical education through rigorous data-driven assessment, targeted curricular improvements, cross-institutional benchmarking, and informed training standards. It evaluates cognitive competence exclusively and should not replace procedural assessment in high-stakes decisions. International validation and the establishment of criterion-related and predictive validity are essential next steps.