Translational Potential and Explainability of Artificial Intelligence-Based Clinical Decision Support for Adults in...

Intensive care units require rapid, high-stakes decision-making. Although artificial intelligence (AI) offers superior predictive accuracy compared with traditional scoring methods, its "black-box" nature remains a barrier to clinical adoption.

This scoping review systematically mapped the translational potential and characteristics of explainable artificial intelligence (XAI) strategies in AI/ML-based clinical decision support tools for adult intensive care unit (ICU) settings.

Following PRISMA-ScR guidelines, we searched Web of Science, PubMed, Scopus, and EBSCOhost up to January 2026. Translational potential was staged using an ICU-adapted, nine-level Technology Readiness Level (TRL) framework, and explainability strategies were classified as post-hoc or inherently interpretable (glass-box) to assess methodological transparency and clinical readiness.

A total of 808 records were identified, of which 29 studies met the inclusion criteria. The findings revealed a marked retrospective predominance (86.2%) and reliance on North American data, predominantly MIMIC (Medical Information Mart for Intensive Care). Tree-based ensembles (82.8%) and post-hoc SHAP explanations (86.2%) were dominant, with proposed clinical utility spanning three domains: therapeutic guidance, resource-allocation optimisation, and user-centric design. Most innovations were standalone, web-based prototypes requiring manual data entry (69.0%, TRL 4-5); a further 10.3% were shared only as open-source code, and only 17.2% reported integration with hospital systems. Only one study claimed clinical maturity (TRL 9), although its validation remained retrospective.

Accuracy is no longer the primary bottleneck; the constraint has shifted to "last-mile" integration and external validity. Current XAI relies almost entirely on post-hoc methods that risk an "illusion of clarity", while inherently interpretable, glass-box models remain a rare but promising alternative. Future research should prioritise external validation in independent settings, prospective evaluation of clinical impact, and explicit comparison between post-hoc and interpretable approaches.