Towards conversational diagnostic artificial intelligence

Area of Science:

• Medical Artificial Intelligence
• Clinical Decision Support Systems
• Natural Language Processing in Healthcare

Background:

• Physician-patient dialogue is crucial for accurate diagnosis and building trust.
• Artificial intelligence (AI) offers potential to enhance healthcare accessibility and quality.
• Developing AI that matches clinical expertise for diagnostic dialogue remains a significant challenge.

Purpose of the Study:

• To introduce AMIE (Articulate Medical Intelligence Explorer), a large language model (LLM)-based AI optimized for diagnostic dialogue.
• To evaluate AMIE's performance against primary care physicians in simulated clinical consultations.
• To assess multiple dimensions of clinical performance, including diagnostic accuracy, communication, and empathy.

Main Methods:

• AMIE was trained using a self-play simulated environment with automated feedback for scalable learning.
• A randomized, double-blind crossover study compared AMIE to 20 primary care physicians using text-based consultations.
• Evaluations were conducted by specialist physicians and patient-actors across 159 diverse case scenarios from Canada, UK, and India.

Main Results:

• AMIE demonstrated superior performance compared to primary care physicians on 30 out of 32 axes evaluated by specialist physicians.
• Patient-actors rated AMIE as superior on 25 out of 26 assessed axes.
• AMIE exhibited greater diagnostic accuracy and enhanced communication and empathy skills.

Conclusions:

• AMIE represents a significant advancement towards conversational diagnostic AI, showing high performance in simulated settings.
• While promising, the study's reliance on text-based chat necessitates further research for real-world clinical translation.
• The findings highlight the potential of LLM-based AI to augment clinical decision-making and patient care.