Integrating retrieval-augmented generation for enhanced personalized physician recommendations in web-based medical...

Area of Science:

• Artificial Intelligence in Healthcare
• Natural Language Processing
• Medical Informatics

Background:

• Web-based medical services enhance healthcare access but struggle with personalized physician recommendations.
• Manual triage by schedulers presents scalability and availability challenges.

Purpose of the Study:

• To develop and validate a Retrieval-Augmented Generation-Based Physician Recommendation (RAGPR) model.
• To improve the accuracy and personalization of physician recommendations in web-based healthcare.

Main Methods:

• Evaluated embedding models (FastText, SBERT, OpenAI) and large language models (Mistral, GPT-4o-mini, GPT-4o) on 646,383 consultation records.
• Assessed model performance using F1-scores and user ratings from triage staff.
• Developed a Retrieval-Augmented Generation-Based Physician Recommendation (RAGPR) model.

Main Results:

• SBERT (95% F1) and OpenAI (96% F1) significantly outperformed FastText (46% F1) in embedding tasks.
• GPT-4o (95% F1) led LLM performance, with Mistral (94% F1) and GPT-4o-mini (92% F1) also showing strong results.
• SBERT and Mistral were identified as optimal due to balanced performance, cost, and implementation ease.

Conclusions:

• The RAGPR model significantly enhances accuracy and personalization in web-based medical services.
• It offers a scalable solution for optimizing patient-physician matching.
• AI-driven recommendations can overcome limitations of manual triage.