Road map for clinicians to develop and evaluate AI predictive models to inform clinical decision-making

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Medicine
• Health Services Research

Background:

• Predictive models have a long history in clinical care, aiding risk assessment and shared decision-making.
• Developing artificial intelligence (AI) predictive models for clinical practice presents unique challenges beyond mere predictive performance.
• Successful integration requires addressing usability and decision-support aspects, not just model accuracy.

Purpose of the Study:

• To outline a nine-stage framework for developing and evaluating predictive AI models for clinical practice.
• To identify potential challenges clinicians may encounter at each stage of AI model development and implementation.
• To provide practical strategies for managing these challenges and facilitating AI adoption.

Main Methods:

• The study details nine stages: clarifying the clinical question, feature selection, dataset selection, model development, validation, interpretation, licensing, maintenance, and impact evaluation.
• Emphasis is placed on the practical aspects of integrating AI predictions into clinical workflows.
• The framework considers multiple interacting components crucial for realizing the benefits of AI prediction models.

Main Results:

• The nine stages encompass the entire lifecycle of an AI predictive model, from conception to post-implementation evaluation.
• Effective clinical integration depends on the accuracy of predictions, user understanding, and the timeliness of information delivery.
• Realizing the benefits of AI models is contingent upon their ability to inform timely and appropriate clinical actions.

Conclusions:

• The impact of AI prediction models on clinical processes and patient outcomes is highly variable.
• Rigorous evaluation using appropriate study designs is essential to understand and optimize the use of AI models in clinical practice.