AgentMD: Empowering language agents for risk prediction with large-scale clinical tool learning

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Healthcare Analytics

Background:

• Clinical calculators are essential but face usability and access challenges.
• Current tools lack automated curation and application capabilities.
• Need for advanced AI to bridge the gap in clinical decision support.

Purpose of the Study:

• To introduce AgentMD, a language agent for curating and applying clinical calculators.
• To evaluate AgentMD's performance in clinical risk prediction and healthcare analytics.
• To demonstrate the potential of language agents in enhancing clinical workflows.

Main Methods:

• AgentMD utilized PubMed to curate 2,164 executable clinical calculators with high accuracy.
• Quality checks and unit tests ensured calculator reliability.
• AgentMD was evaluated as a tool user for autonomous calculator selection and application.

Main Results:

• AgentMD demonstrated superior accuracy (87.7%) in risk prediction compared to GPT-4 (40.9%).
• Accurate individual-level medical risk computation was confirmed on 698 emergency department notes.
• AgentMD provided population-level insights for institutional risk management.

Conclusions:

• Language agents like AgentMD can effectively curate and utilize clinical calculators.
• AgentMD offers significant improvements for individual patient risk assessment.
• AgentMD supports at-scale healthcare analytics and institutional risk management.