Engineering framework for curiosity-driven and humble AI in clinical decision support

Area of Science:

• Artificial Intelligence
• Clinical Decision Support
• Medical Informatics

Background:

• Large language models (LLMs) in clinical AI often exhibit overconfidence, lacking genuine medical understanding.
• This overconfidence stems from conflating statistical pattern recognition with true clinical reasoning.
• There is a need for AI systems that demonstrate appropriate uncertainty and information-seeking behavior.

Purpose of the Study:

• To introduce BODHI (Balanced, Open-minded, Diagnostic, Humble, and Inquisitive), an engineering framework for developing humble and curiosity-driven clinical AI.
• To address the limitations of LLMs in expressing appropriate confidence and epistemic uncertainty.
• To constrain LLM responses using virtue-based rules and uncertainty decomposition.

Main Methods:

• Developed BODHI, a dual reflective architecture for AI systems.
• Implemented uncertainty decomposition into task-specific dimensions.
• Constrained model responses using virtue-based stance rules derived from a Virtue Activation Matrix.
• Validated the framework on 200 clinical vignettes using GPT-4o-mini and GPT-4.1-mini.

Main Results:

• BODHI significantly improved overall clinical response quality for both GPT-4.1-mini (+16.6 pp) and GPT-4o-mini (+2.2 pp).
• The framework achieved very large effect sizes for curiosity (context-seeking rate) and humility (hedging) metrics.
• Appropriate clarifying questions increased dramatically, from 7.8% (GPT-4.1-mini) and 0.0% (GPT-4o-mini) at baseline to 97.3% and 73.5% respectively.

Conclusions:

• LLMs can be reliably constrained to operate within epistemic boundaries using structured uncertainty decomposition and virtue-aligned rules.
• BODHI offers a viable pathway towards safer and more reliable clinical AI deployment.
• The framework effectively promotes information-gathering behavior in AI systems, crucial for clinical settings.