Development of a Liver Disease-Specific Large Language Model Chat Interface using Retrieval Augmented Generation

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Natural Language Processing
• Biomedical Informatics

Background:

• Commercial large language models (LLMs) lack clinical optimization and may generate inaccurate information.
• Retrieval-augmented generation (RAG) enhances LLMs by incorporating custom data, aiming to reduce hallucinations.
• Developing specialized LLMs is crucial for reliable clinical information processing.

Approach:

• Created "LiVersa," a liver disease-specific LLM using a Protected Health Information (PHI)-compliant platform.
• Employed RAG to integrate 30 American Association for the Study of Liver Diseases (AASLD) guidelines into LiVersa.
• Evaluated LiVersa's performance against human trainees on hepatitis B and hepatocellular carcinoma knowledge assessments.

Key Points:

• LiVersa achieved 100% accuracy in binary (yes/no) responses for clinical questions.
• Detailed explanations from LiVersa were not fully accurate for three questions.
• The study demonstrates the feasibility of creating disease-specific, PHI-compliant LLMs via RAG.

Conclusions:

• LiVersa serves as a proof-of-concept for RAG-customized clinical LLMs.
• Disease-specific LLMs can enhance specificity in clinical domains like hepatology.
• This approach offers a potential pathway toward personalized medicine through tailored AI tools.