Large Language Models and Primary Care: A Scoping Review

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Primary Care Research

Background:

• Large Language Models (LLMs) present significant opportunities for enhancing primary care services.
• However, concerns regarding their reliability and potential biases necessitate a thorough review of existing evidence.

Purpose of the Study:

• To conduct a scoping review of Large Language Model (LLM) applications in primary care.
• To synthesize evidence on LLM utility, performance, and associated risks, adhering to PRISMA-ScR guidelines.

Main Methods:

• Searched 10 databases for original research on LLMs in primary care.
• Assessed risk of bias using the PROBAST tool.
• Included 28 studies in the final analysis.

Main Results:

• Common LLM applications included information extraction (25%) and predictive modeling (14%).
• Performance was high for structured tasks (F1-scores 0.70-0.95) but variable for interpretive tasks.
• A significant proportion of studies (39%) had a high risk of bias, with only 18% showing low risk.
• Most studies originated from high-income countries, underrepresenting LMIC regions.

Conclusions:

• LLMs show utility in primary care for administrative and clinical tasks.
• Geographic skew and methodological limitations, including high risk of bias, impede widespread adoption.
• Rigorous, inclusive evaluation frameworks are essential before the broad implementation of LLMs in primary care.