Generative AI and unstructured audio data for precision public health

Area of Science:

• Computational epidemiology
• Artificial intelligence in healthcare
• Infectious disease modeling

Background:

• Early pandemic variant detection is crucial for public health interventions.
• Traditional methods rely on genetic sequencing, which can be time-consuming.
• Subtle changes in symptomatology may serve as early biomarkers for new variants.

Purpose of the Study:

• To evaluate the efficacy of large language models (LLMs) in classifying COVID-19 variants based on patient-reported symptoms.
• To assess the utility of LLM-generated summaries of patient experiences for variant prediction.
• To compare LLM-based variant classification with traditional symptom data analysis.

Main Methods:

• Transcribed videos of personal COVID-19 experiences were processed using the o1 LLM.
• LLM summaries excluded non-symptomatic data (dates, vaccinations, testing) to simulate early pandemic conditions.
• A neural network was trained on LLM summaries to predict 'Omicron' vs. 'Pre-Omicron' variants.

Main Results:

• The LLM-trained neural network achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.823.
• A comparative model trained on binary symptom data achieved a lower AUROC of 0.769.
• LLM analysis of patient narratives demonstrated superior performance in variant classification.

Conclusions:

• LLMs can effectively identify subtle symptomatological shifts indicative of new COVID-19 variants.
• LLM-derived insights from patient experiences offer a valuable tool for early pandemic surveillance.
• This approach highlights the potential of integrating LLMs and audio data into future pandemic management strategies.