Temporal maintenance of an adolescent suicide attempt screening score in national surveys: An observational study

Screening scores may retain risk ranking while their probabilities become inaccurate in later populations. The information needed to repair this temporal drift is rarely quantified for adolescent suicide screening. We analyzed 5 independent national Youth Risk Behavior Survey cross sections from 2015 through 2023. Of 81,401 sampled students, 68,682 had observed past year attempt status. A survey weighted 2015 ridge score based on 14 harmonized predictors was frozen and transported to 4 later surveys. For each later survey, 50 repeated 5-fold evaluations that separated primary sampling units compared no update, intercept recalibration, intercept and slope recalibration, and full refitting of the same specification using 250 to 4000 target labels. Direct suicide items and identity attributes were excluded. The analysis included 6855 reported attempts. Frozen score areas under the receiver operating characteristic curve remained 0.852 to 0.860, but average risk was overestimated in 2017, 2019, and 2023. Predictor distribution and conditional model components opposed each other in 3 surveys. With 1000 labels, intercept recalibration met a prespecified descriptive calibration rule in 76 to 88% of repetitions; rates were 92 to 100% with 2000 labels. At 250 labels, refitting reduced median area under the curve by 0.036 to 0.043 and increased Brier score by 0.0047 to 0.0060. Intercept recalibration outperformed refitting in all 50 paired repetitions. Alternative ridge penalties did not remove this instability. Stable discrimination did not ensure stable absolute risk. Intercept recalibration was more reliable than coefficient refitting with scarce new outcomes. Maintenance requires reserved current data and an update strategy matched to the observed drift.