What a Near-Perfect AUROC Does Not Show: Random-Panel and Label-Permutation Controls for a Sepsis Blood-Transcriptome...

Sepsis blood-transcriptome signatures are usually judged against healthy controls. Two questions are rarely separated: whether the classifier exceeds chance, and whether its genes outperform an arbitrary list of equal size. We tested both on public microarray data (discovery GSE65682, n = 359; validation GSE57065, n = 53), using the 9983 of 11,222 genes measurable on both platforms. A 10-gene signature was selected by nested cross-validation and applied to validation without refitting. It separated sepsis from health perfectly (area under the receiver operating characteristic curve, AUROC, 1.000). The pipeline exceeded chance: no label permutation in 1000 matched it (p = 0.001). The genes did not outperform random lists: 12.5% of 2000 random 10-gene panels matched or exceeded the signature, as did 67.6% of panels drawn from the same 300 preselected candidates. Against non-infected intensive care patients, cross-fitted AUROC fell to 0.730 (95% confidence interval 0.642-0.818), unchanged after age and sex matching. At a threshold of 0.5, the cross-fitted model called 70 of these 75 patients septic. Selection was unstable: 200 bootstrap resamples gave a median of 6 genes, from 109 distinct genes. A near-perfect AUROC against healthy controls reflects an easy benchmark, not a specific gene list.