Inter-rater reliability of needle EMG pattern classification among physicians

To evaluate the inter-rater reliability of needle electromyographic (EMG) assessment among 19 physicians with varying clinical experience and specialties using a consensus standard as reference.

The study followed the GRRAS guidelines. First, four EMG experts agreed on classifying thirty EMG recordings as normal, myogenic, or neurogenic and referred to as a consensus standard (CS). Second, those anonymised recordings were independently classified by 19 non-expert physicians. Thereafter, Cohen's kappa (κ) was calculated for each physician against the CS. Fleiss' κ was used for multi-rater agreement. Category-specific (one-vs-rest) κ, sensitivity and specificity were computed. Subgroup analyses by experience and specialty were performed. Post-hoc power analysis was used for the specialty comparison.

The mean κ versus the consensus standard across all physicians was 0.688 (median 0.747, SD 0.172; range 0.287-0.900). Overall multi-rater agreement (Fleiss' κ) was 0.581. Category-specific κ values were: neurogenic 0.723, normal 0.368, myogenic 0.334. Sensitivity/specificity: neurogenic 94.6%/87.1%, myogenic 69.7%/96.1%, normal 78.2%/88.9%. Agreement with the CS increased with experience: mean κ rose from 0.365 (group 1, ≤1 year) to 0.798 (group 4, >10 years). No significant difference was found between neurologists and clinical physiologists (Mann-Whitney U, p = 0.174); post-hoc power was low (≈45%), thus non-significance should not be interpreted as equivalence. EMG experts showed almost perfect agreement (κ = 0.85-0.95).

In this exploratory study, physicians demonstrated moderate to substantial agreement with a CS standard in EMG classification. Neurogenic changes are identified most reliably, whereas differentiation between normal and myogenic changes remains challenging. Greater experience was associated with better agreement in our sample, but this finding requires confirmation in larger studies. No statistically significant difference was found between neurologists and clinical physiologists, although the analysis was underpowered. Individual competency varied widely within specialties, suggesting that objective, competency-based qualification may be more relevant than specialty alone.