Counterfactuals and uncertainty-based explainable paradigm for the automated detection and segmentation of renal...

Contrast-enhanced abdominal computed tomography (CT) scans frequently detect renal cysts, some of which may be malignant. Early and precise localization improves patient management and enables quantitative image analysis. However, current segmentation methods often lack interpretability at both feature and pixel levels, highlighting the need for an explainable approach that can detect and correct model errors. We developed an interpretable 3D renal cyst segmentation framework and validated it on 1486 cysts from 568 patients across three centers and an open-source dataset. The framework first isolated the kidney and then segmented renal cysts, integrating segmentation quality control via uncertainty estimation. A Variational Autoencoder Generative Adversarial Network (VAE-GAN) learned latent representations of 3D patches and reconstructed inputs. Latent-space modifications, guided by segmentation model gradients, generated counterfactual explanations corresponding to varying Dice similarity coefficients (DSC). Radiomics features extracted from counterfactual images, using ground-truth masks, were analyzed for associations with segmentation performance. The framework achieved a DSC of 0.82 on two external test sets, with sensitivities of 0.95 and 0.97, respectively. DSCs for original and VAE-GAN-reconstructed images showed no significant differences. Counterfactual explanations revealed how changes in cyst appearance influenced model outputs and exposed segmentation discrepancies. Radiomics features positively and negatively correlated with DSCs were identified. Using uncertainty estimates to flag segmentations with DSC < 0.75 reduced poor segmentations from 17% to 5% when only 20% of cases were sent for manual correction. This combination of counterfactual explanations and uncertainty maps improved segmentation interpretability and can be generalized for other applications.