Ordinal Gaussian Processes for Gleason grading: Enhancing performance and calibration across foundation model rankability scores

Gleason grading of prostate cancer is a critical clinical task inherently defined by an ordinal progression of tissue deterioration. However, most automated computational approaches simplify this problem into a standard multi-class classification problem, which ignores its ordinal structure. Furthermore, the suitability of foundation models-which currently provide state-of-the-art latent representations of histological images- for ordinal decision making remains underexplored. Specifically, it is unclear how model design should adapt when these embeddings fail to naturally capture disease progression.

In this context, we investigate the concept of rankability, defined as the degree to which the embedding space of a foundation model inherently preserves the natural ordering of Gleason grades. We systematically analyze how this property impacts downstream ordinal predictions. Consequently, we propose a novel framework that integrates pre-trained foundation model embeddings with Ordinal Gaussian Processes (OGPs). By explicitly incorporating an ordinal likelihood, our approach mathematically enforces the strictly ordered nature of disease severity within the probabilistic model.

Evaluated across two publicly available prostate cancer datasets exhibiting different levels of rankability, our experiments show the benefits of OGPs for ordinal problems. While both ordinal and standard multi-class approaches achieve similar performance in naturally highly rankable embedding spaces, the proposed OGP framework significantly outperforms its non-ordinal counterparts in low-rankability scenarios. In these challenging settings, explicitly integrating an ordinal likelihood yields substantial improvements in both predictive agreement metrics and calibration.

Our findings highlight that explicitly modeling ordinality acts as a crucial safeguard in automated grading systems. When state-of-the-art foundation models fail to capture the sequential nature of disease progression, integrating an ordinal likelihood improves the predictive robustness and clinical reliability necessary for deploying computational pathology tools in practice.