Machine Learning Improves Questionnaire-Based Sleep Apnea Screening: A Multi-Cohort Multi-Visit Validation Study

Obstructive sleep apnea (OSA) affects a large portion of the global population but remains substantially underdiagnosed due to limited access to polysomnography. While machine learning has shown promise for scalable OSA screening, many recent models rely on overnight physiological monitoring and are not validated across heterogeneous cohorts. This study develops and evaluates questionnaire-based CatBoost models for OSA detection using two large, multi-cohort datasets (SHHS and WSC). The models use 42 easily obtainable features that require no specialized equipment, which could potentially enable highly accessible screening. We assessed performance through comprehensive internal and external validation, including within-cohort external testing (SHHS1 $\to$ SHHS2) and cross-cohort generalization (SHHS1 $\to$ WSC1), across three clinically relevant AHI cutoffs ( $\ge$ 5, $\ge$ 15, $\ge$ 30 events/h). In internal validation, models achieved consistently balanced performance with AUC 0.74-0.76, sensitivity 67%-73%, and specificity 62%-70% across all cutoffs. Performance remained stable during within-cohort external testing at cutoff $\ge$ 15 (AUC 0.75, sensitivity 71%, specificity 65%). However, cross-cohort validation revealed noticeable performance degradation due to population distribution shifts, most pronounced at $\ge$ 5 where specificity dropped substantially (48.5% $\pm$ 4.7%) when models trained on the older SHHS cohort were applied to the younger WSC population. SHAP analysis confirmed that predictions relied primarily on established clinical risk factors including neck circumference, age, BMI, and symptom history. These results demonstrate that questionnaire-based machine learning can achieve more balanced sensitivity-specificity trade-offs than traditional scoring rules, and highlight the importance of multi-cohort external testing.