Reinforcement learning for individualized lung cancer screening schedules: A nested case-control study

The current guidelines for managing screen-detected pulmonary nodules offer rule-based recommendations for immediate diagnostic work-up or follow-up at intervals of 3, 6, or 12 months. Customized visit plans are lacking.

To develop individualized screening schedules using reinforcement learning (RL) and evaluate the effectiveness of RL-based policy models.

Using a nested case-control design, we retrospectively identified 308 patients with cancer who had positive screening results in at least two screening rounds in the National Lung Screening Trial. We established a control group that included cancer-free patients with nodules, matched (1:1) according to the year of cancer diagnosis. By generating 10,164 sequence decision episodes, we trained RL-based policy models, incorporating nodule diameter alone, combined with nodule appearance (attenuation and margin) and/or patient information (age, sex, smoking status, pack-years, and family history). We calculated rates of misdiagnosis, missed diagnosis, and delayed diagnosis, and compared the performance of RL-based policy models with rule-based follow-up protocols (National Comprehensive Cancer Network guideline; China Guideline for the Screening and Early Detection of Lung Cancer).

We identified significant interactions between certain variables (e.g., nodule shape and patient smoking pack-years, beyond those considered in guideline protocols) and the selection of follow-up testing intervals, thereby impacting the quality of the decision sequence. In validation, one RL-based policy model achieved rates of 12.3% for misdiagnosis, 9.7% for missed diagnosis, and 11.7% for delayed diagnosis. Compared with the two rule-based protocols, the three best-performing RL-based policy models consistently demonstrated optimal performance for specific patient subgroups based on disease characteristics (benign or malignant), nodule phenotypes (size, shape, and attenuation), and individual attributes.

This study highlights the potential of using an RL-based approach that is both clinically interpretable and performance-robust to develop personalized lung cancer screening schedules. Our findings present opportunities for enhancing the current cancer screening system.