Human-AI co-design for clinical prediction models

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Health Data Science

Background:

• Developing clinical prediction models (CPMs) demands extensive collaboration.
• Unstructured clinical notes present a vast, complex data source for modeling.
• Existing methods struggle with the scale and interpretability of CPMs derived from notes.

Purpose of the Study:

• Introduce HACHI, a human-in-the-loop framework for developing interpretable CPMs from clinical notes.
• Accelerate the creation of transparent and steerable CPMs through AI-expert collaboration.
• Improve the generalizability and clinical relevance of predictive models.

Main Methods:

• HACHI framework: iterative process alternating AI agent exploration and expert feedback.
• AI agent utilizes statistical tools and embedded knowledge to identify predictive concepts.
• Domain experts guide the AI, refining concepts and ensuring model transparency.
• CPMs are defined as linear models based on yes-no questions derived from clinical notes.

Main Results:

• HACHI outperforms existing approaches in real-world prediction tasks (acute kidney injury, traumatic brain injury).
• The framework discovers clinically relevant concepts and enhances model generalizability across sites and time.
• Demonstrates improved interpretability and transparency in the developed CPMs.

Conclusions:

• HACHI effectively accelerates the development of interpretable clinical prediction models from unstructured data.
• Human-in-the-loop AI frameworks are crucial for optimizing collaboration and model performance.
• Emphasizes the importance of human oversight in guiding AI for clinical applications, addressing bias and leakage.