Large Language Models for Epidemiological Research via Automated Machine Learning: Case Study Using Data From the...

Area of Science:

• Epidemiological research
• Natural Language Processing (NLP)
• Machine Learning

Background:

• Large language models (LLMs) have advanced Natural Language Processing (NLP).
• Current NLP applications in epidemiology are mainly limited to analyzing electronic health records and social media data.
• The potential of LLMs in broader epidemiological contexts remains underexplored.

Purpose of the Study:

• To develop and evaluate LLM-based prediction models using text data from an epidemiological cohort.
• To compare the performance of LLM prediction models against classical regression methods.
• To explore the utility of NLP beyond traditional data sources in epidemiological research.

Main Methods:

• Utilized data from the British National Child Development Study (10,567 children).
• Fine-tuned pretrained language models to predict reading comprehension, Body Mass Index (BMI), and physical activity at age 33.
• Compared NLP model performance against linear and logistic regression models using demographic and lifestyle factors.

Main Results:

• NLP models significantly outperformed linear regression in predicting reading comprehension scores.
• Predictive performance for physical activity was comparable between NLP and regression methods, performing slightly better than random chance.
• Linear regression models demonstrated superior performance in predicting BMI compared to NLP approaches.
• NLP models did not show improvement over baseline mean prediction for BMI.

Conclusions:

• LLMs demonstrate potential for analyzing text data in epidemiological studies.
• The effectiveness of LLMs is contingent on the direct relevance of the text's topic to the outcome being predicted.
• Future epidemiological studies should consider incorporating open-ended questions and NLP methods to capture health concepts and lived experiences.