Generative Large Language Model-Powered Conversational AI App for Personalized Risk Assessment: Case Study in COVID-19
View abstract on PubMed
Summary
This summary is machine-generated.Generative large language models (LLMs) offer a no-code solution for real-time disease risk assessment, outperforming traditional methods in low-data scenarios. This AI approach enhances clinical decision-making with conversational interactions.
Area Of Science
- Artificial Intelligence in Healthcare
- Natural Language Processing
- Machine Learning for Clinical Decision Support
Background
- Large language models (LLMs) show promise for healthcare tasks like disease risk assessment, offering flexibility beyond traditional structured data methods.
- Generative LLMs can be applied to clinical settings without requiring programming expertise.
- Conversational AI powered by LLMs presents a novel approach to disease risk stratification.
Purpose Of The Study
- To evaluate generative LLMs (LLaMA2-7b, Flan-T5-xl) for predicting COVID-19 severity.
- To develop a real-time, no-code risk assessment solution using chatbot interactions.
- To compare LLM performance against traditional machine learning classifiers (logistic regression, XGBoost, random forest).
Main Methods
- Fine-tuning LLMs with few-shot natural language examples from a pediatric COVID-19 dataset.
- Developing a mobile application for real-time, no-code risk assessment via clinician-patient dialogue.
- Comparing LLM performance with traditional classifiers using Area Under the Curve (AUC) and analyzing LLM attention layers for interpretability.
Main Results
- Generative LLMs demonstrated strong performance, particularly in low-data settings (e.g., T0-3b-T achieved AUC 0.75 in zero-shot).
- LLMs showed competitive or superior performance compared to traditional models as data increased (e.g., Flan-T5-xl-T reached AUC 0.70 at 32-shot).
- The mobile app provided real-time assessments and personalized insights through attention-based feature importance.
Conclusions
- Generative LLMs are a robust alternative to traditional classifiers, especially with limited labeled data.
- LLM-powered conversational AI offers adaptability for clinical settings, enabling real-time, personalized assessments without coding.
- Further research into LLM applications for disease risk assessment and clinical decision support is warranted.