Current Use And Evaluation Of Artificial Intelligence And Predictive Models In US Hospitals

Area of Science:

• Health Informatics
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• The increasing use of artificial intelligence (AI) and machine learning (ML) in healthcare necessitates robust evaluation frameworks.
• Ensuring predictive models are fair, appropriate, valid, effective, and safe (FAVES) is crucial for patient outcomes and equitable care.
• Current governance practices for AI/ML models in hospitals require examination.

Purpose of the Study:

• To analyze the utilization and evaluation methods of AI and predictive models in US hospitals.
• To identify how hospitals assess the accuracy and bias of these models.
• To understand factors influencing hospitals' evaluation practices for predictive models.

Main Methods:

• Analysis of data from the 2023 American Hospital Association Annual Survey Information Technology Supplement.
• Examination of reported practices for using and evaluating AI and predictive models.
• Statistical analysis to identify factors associated with model evaluation.

Main Results:

• 65% of US hospitals utilize predictive models, with 79% sourcing them from electronic health record developers.
• 61% of hospitals evaluate model accuracy locally, but only 44% report local bias evaluation.
• Hospitals developing their own models, possessing high operating margins, or belonging to health systems were more likely to conduct local evaluations.

Conclusions:

• A significant gap exists in the evaluation of bias for predictive models across US hospitals.
• Developing and implementing standardized tools and support for assessing FAVES principles is essential.
• Policy interventions are needed to promote safe and equitable AI adoption, preventing a digital divide in healthcare AI.