Characterizing the clinical profile of primary headache using explainable artificial intelligence

Background/HypothesisDifferentiating primary from secondary headaches remains challenging due to overlapping symptoms and the subjective nature of pain assessment. Machine learning (ML) combined with explainable artificial intelligence (XAI) offers a promising approach for improving diagnostic accuracy while maintaining clinical interpretability. The study aimed to develop an interpretable ML model to differentiate primary from secondary headaches and to identify the relative diagnostic contributions of individual clinical features.MethodsThis cross-sectional study analyzed 1219 patients using routine clinical history variables, including 21 triggers, 15 associated symptoms, temporal patterns, pain intensity (visual analog scale [VAS]), and demographic factors. Data were obtained from the Global Migraine and Pain Society (GMPS) registry and the epikriz.com/headache platform, restricted to patients managed by specialist headache neurologists at two headache clinics in Türkiye. The binary classification outcome was clinically assigned headache label: primary versus secondary headache disorder (primary headache designated as the positive class). An eXtreme Gradient Boosting (XGBoost) classifier was developed with Random Over-Sampling Examples to address class imbalance, which was applied exclusively to the training partition. Model interpretability was achieved using SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME).ResultsThe model demonstrated excellent discrimination (AUC = 0.905), with sensitivity of 90.8%, specificity of 87.5%, precision of 99.7%, and an F1-score of 0.950. Associated symptom burden was the strongest predictor of primary headache classification (mean absolute SHAP = 1.075), driven principally by nausea, phonophobia, and photophobia. Trigger burden ranked second (mean absolute SHAP = 0.790), with stress, hunger, and odors most prevalent. Two laterality features independently discriminated headache type: strictly unilateral side-shifting (mean absolute SHAP = 0.545) and unilateral-to-bilateral transition (mean absolute SHAP = 0.481), both favoring primary classification. Higher diurnal pattern scores were associated with secondary headache disorders (mean absolute SHAP = 0.257). VAS contributed minimally (mean absolute SHAP = 0.101), suggesting that the syndromic profile outperforms pain severity as a diagnostic discriminator in this cohort.Conclusion/InterpretationIn this cohort, an ML model incorporating trigger profiles, associated symptoms, and temporal headache characteristics demonstrated good discrimination between clinically assigned primary and secondary headache labels. Associated symptom burden and trigger burden were more influential in model classification than pain intensity. This finding aligns with clinical practice, where headache diagnosis relies on symptom patterns and triggers rather than pain intensity alone. XAI frameworks may help make such classification models more clinically interpretable, although further external validation is required.