Uncertainty-Aware Selective Prediction in Brain-Tumour MRI Classification: Selective Risk Reduction and...

Automated brain-MRI analysis is usually judged by accuracy at a single operating point, yet a model that is accurate on average may still fail without signalling that it is about to fail. Selective prediction addresses this by allowing a model to abstain, but the resulting trade-off between coverage and risk is rarely quantified in medical imaging.

This study asks whether confidence- and entropy-based selective prediction produces a measurable reduction in selective risk in a biomedical vision-language classifier by concentrating errors among withheld cases, and at what coverage cost. Three further components of a brain-MRI-analysis framework-volumetric segmentation, glioma grading and structured report generation-are evaluated as complementary evidence about the reliability profiles of different modules, not as independent contributions.

A LoRA-fine-tuned LLaVA-Med classifier was evaluated on the 1000-image BRISC2025 test partition, with a disjunctive rejection rule based on softmax confidence and predictive entropy. Volumetric segmentation used nnU-Net v2 (3D full-resolution) on BraTS2020 with a three-fold ensemble evaluated on 74 held-out cases; glioma grading (high- versus low-grade) used DenseNet121 on multi-slice representations; report generation used Meta Llama 3.2 3B Instruct fine-tuned with QLoRA on structured findings. All proportions are reported with 95% Wilson score intervals under a single analysis plan documented before the reported analyses were finalised.

Withholding 1.80% of predictions (95% CI 1.14-2.83) raised selective accuracy to 97.76% (95% CI 96.63-98.52) and reduced selective risk from 3.40% to 2.24%, a relative reduction of 34.1%. Withheld predictions were strongly enriched for errors (66.7% versus 2.24%; odds ratio 87.3, Fisher exact p = 4.6 × 10^-15), capturing 35.3% of all errors while issuing 99.38% of the predictions that were correct. Considered as a detector of its own errors, the rule has a balanced accuracy of 67.34%; this describes the binary rejection decision at the realised threshold and does not identify the AUROC of the underlying continuous score, which the single operating point leaves anywhere in [0.351, 0.996]. Because the per-sample probability vectors were not retained, the full risk-coverage curve, the rule ablations and calibration error could not be computed, and the thresholds (τ_c = 0.6, τ_H = 1.0) were selected empirically on an undocumented partition; the selective-prediction finding is therefore reported as a single-operating-point case study and is exploratory. Among the complementary components, three-fold ensembling improved held-out mean Dice from 0.7948 to 0.8150 (+0.0202), reported descriptively because the per-case values needed for a paired test were not preserved. Glioma grading did not achieve better-than-chance discrimination: balanced accuracy 49.91%, Cohen's κ -0.0023, Matthews correlation coefficient -0.0026, low-grade recall 8.96% (95% CI 5.81-13.57), and accuracy (74.03%) below a constant majority-class classifier (79.46%); the associated McNemar statistic is reported descriptively because slice-level observations are not independent. Report generation produced high automatic similarity scores (BERTScore F1 0.9513) that are not interpretable without baselines.

Confidence- and entropy-based abstention produced a measurable reduction in selective risk at low rejection rates, and it did so because errors concentrated among the cases were withheld by the confidence/entropy rule rather than because predictions were discarded indiscriminately. At the realised coverage the rule captured 65.5% (95% bootstrap CI 41.9-87.8) of the reduction achievable by an oracle that rejects only true errors. The component-wise comparison shows that this reliability behaviour cannot be assumed for every module: high aggregate accuracy and useful discrimination came apart in the grading task. These findings are component-wise and do not constitute an end-to-end, externally validated or clinically validated pipeline. The vision-language evaluation is at image level: BRISC2025 carries no patient identifiers, so patient-level independence cannot be established and the reported intervals should be read with that constraint in mind.