Reject Option for Medical Image Classification Using CNN-Derived Latent-Space Neighbourhood Analysis

Reliable uncertainty estimation is a key requirement for the clinical adoption of convolutional neural networks (CNNs) in biomedical imaging. We propose a reject option framework that assesses prediction reliability by leveraging the local structure of the latent space. The method is based on three latent-space features: Simple Data Point Target Density ( ${\mathrm{DPTD}}_S$ ), defined as the proportion of a sample's nearest neighbours sharing its label, a distance-weighted extension ( ${\mathrm{DPTD}}_W$ ), and the $k_{}^{\ast }$ value. These metrics characterise local label agreement and intra-class consistency without relying on explicit parametric assumptions. A logistic regression rejector is trained post hoc using features derived from the latent representations of the base CNN. Experiments were conducted on two public neuroimaging datasets: ADNI (FDG-PET for Alzheimer's disease) and PPMI (DaTSCAN SPECT for Parkinson's disease). Using all three latent-space features, the rejector achieved AUC-ROC values of 0.843 (95% CI: 0.809-0.874) on ADNI and 0.884 (95% CI: 0.867-0.900) on PPMI, with corresponding eAURC values of 0.0278 (95% CI: 0.0193-0.0370) and 0.0169 (95% CI: 0.0133-0.0209), respectively. For the full-feature configuration, no significant difference from maximum softmax probability (MSP) was detected on ADNI for either AUC-ROC or eAURC, whereas MSP performed significantly better on PPMI for both metrics. The proposed framework provides a representation-based mechanism to assess prediction reliability through local latent-space structure. As a post-hoc approach, it can be integrated into existing CNN pipelines without modifying the underlying model, offering a practical tool for uncertainty-aware decision support in biomedical imaging.