Deep sequence learning with multi-task supervision for scalable population health monitoring

Models designed for artificial intelligence-driven population health monitoring must be able to integrate multiple types of information over time, be heterogeneous in the data they consider, and be deployed at scale. This research proposes an integrated multitask framework for predicting population disease incidence and mortality risk, based on deep learning and using both longitudinal biobank and National Health Survey data in a scalable manner. The framework will learn shared temporal representations from clinical lab-generated/demographically defined survey variables by applying a strict prospective evaluation approach across the framework. On UK Biobank, the proposed model achieves an AUROC of 0.842 and a C-index of 0.791, outperforming strong baselines such as Proformer having AUROC 0.829 and UKB-MDRMF having AUROC 0.821. On NHANES, the model achieves an AUROC of 0.861 and a C-index of 0.808, exceeding those of commonly used boosted tree and survival-based approaches. Temporal robustness analysis shows controlled performance degradation across extended horizons, with UK Biobank AUROC decreasing from 0.872 at 1 year to 0.819 at 10 years and NHANES AUROC decreasing from 0.886 at 1 year to 0.836 at 5 years. Subgroup evaluation demonstrates stable performance across age, sex, and clinical risk strata, while scalability analysis confirms near-linear training growth and stable inference latency below three milliseconds per individual at biobank scale.