Clinically-guided models or foundation models? predicting cervical spondylotic myelopathy from electronic health...

Cervical spondylotic myelopathy (CSM) is the leading cause of spinal cord dysfunction in older adults, yet diagnosis is frequently delayed due to insidious symptom onset and limited clinical recognition. We developed and externally validated machine learning models using structured electronic health record (EHR) data to predict incident CSM diagnoses up to 30 months in advance. Using data from ~2 million patients in the Merative™ MarketScan® claims database and our institutional EHR, we evaluated a spectrum of modeling strategies, ranging from simple, clinically guided architectures to large-scale pretrained foundation models. These included count-based feed-forward networks, a clinically curated Mamba state-space model, two mid-scale transformer models (CoreBEHRT and CEHRBERT), and large foundation models (clmbr-t-base and clmbr-t-5k-CSM). While large foundation models achieved an overall stronger performance during internal validation in the larger, more heterogeneous dataset, the clinically oriented models generalized more effectively in external validation across a separate health system. These findings underscore the promise of foundation models in capturing rich EHR representations yet highlight persistent challenges in their generalizability. In contrast, domain-informed models, despite their simplicity, may offer greater robustness across care settings.