A Not So Perfect Score: Factors Associated with the Rate of Straight Line Scoring in Oncology Training Programs

Straight line scoring (SLS), defined as trainee assessments with the same score for all evaluation items, is statistically improbable and potentially indicates inaccurate assessment. Factors contributing to higher SLS rates are unknown, and knowledge of SLS prevalence within oncologic training is lacking. SLS frequency was measured for evaluations from all Accreditation Council for Graduate Medical Education (ACGME)-accredited programs at a single cancer care institution between 2014 and 2018. SLS prevalence was estimated using hierarchical linear models (HLM) that considered characteristics of evaluator, trainee, and evaluation potentially related to SLS. Results were compared with national SLS rates. Six thousand one hundred sixty evaluations were included from 476 evaluators. Overall prevalence of SLS was 12.1% (95% CI 4.5-28.8). Residents (vs fellows) were less likely to have SLS evaluations (OR 0.5, 95% CI 0.4-0.8), though for all trainees increasing training year corresponded with increasing SLS frequency (OR 1.5, 95% CI 1.3-1.7). SLS was more common in procedural specialties compared with medical specialties (OR 2.1, 95% CI 1.1-3.8). Formative evaluations had lower SLS rates (OR 0.6, 95% CI 0.5-0.9) than summative evaluations, while milestone-based evaluations had higher rates than those that were not milestone-based (OR 1.5, 95% CI 1.03-2.2). Features of evaluators, such as subspecialty within oncology, and of trainees, such as seniority or trainee type, were related to SLS. Summative intent and milestone-based evaluations were more likely to be straight line scored. Specific evaluation scenarios at higher risk of SLS should be further examined.