SHORTKIT-ML: A UNIFIED MULTI-PERSPECTIVE FRAMEWORK FOR DETECTING SHORTCUT LEARNING IN MEDICAL IMAGING EMBEDDINGS

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Imaging

Background:

• Shortcut learning in clinical AI leads to biased predictions by exploiting spurious correlations instead of genuine clinical features.
• Existing methods for detecting these shortcuts are fragmented and lack standardized evaluation across diverse datasets and AI models.
• This necessitates a unified approach to systematically analyze and address shortcut learning in clinical artificial intelligence.

Purpose of the Study:

• To introduce ShortKit-ML, an open-source Python framework designed for comprehensive shortcut analysis within embedding spaces.
• To provide a modular pipeline integrating over 20 detection methods and six mitigation strategies for unified shortcut auditing.
• To facilitate systematic evaluation of shortcut learning across various datasets and model architectures in clinical AI.

Main Methods:

• Developed ShortKit-ML, a Python framework for shortcut analysis in embedding spaces, integrating diverse detection and mitigation techniques.
• Implemented a modular pipeline covering embedding analysis, fairness metrics, training dynamics, causal inference, explainability, and representation analysis.
• Evaluated the framework on chest X-ray datasets (CheXpert, MIMIC-CXR), synthetic benchmarks, and a cross-domain dataset (CelebA).

Main Results:

• Multi-method auditing within ShortKit-ML provides more stable and interpretable evidence of shortcuts compared to single methods.
• Detector disagreement analysis effectively reveals meaningful differences in model representations.
• The framework demonstrated robust performance across various datasets, highlighting its generalizability.

Conclusions:

• ShortKit-ML offers a unified and systematic approach to auditing shortcut learning in clinical AI, enhancing model reliability and fairness.
• The framework's integrated methods and automated reporting provide valuable insights into model behavior and potential biases.
• Public availability of the source code and documentation promotes wider adoption and further research in trustworthy AI for healthcare.