Extraction of Endoscopic Markers from Clinical Notes in Italian Patients with Autoimmune Atrophic Gastritis Using...

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing for Healthcare
• Computational Linguistics

Background:

• Large Language Models (LLMs) show promise for clinical applications but face challenges like data privacy, computational costs, and hallucinations.
• Small Language Models (SLMs) offer a potential solution for efficient, secure, on-device processing in healthcare.

Purpose of the Study:

• To evaluate the efficacy of a locally deployed, task-specific Small Language Model (SLM) for extracting clinical information.
• To assess the performance of a fine-tuned IT5 model in identifying endoscopic markers for Autoimmune Atrophic Gastritis (AAG) from Italian clinical notes.

Main Methods:

• Fine-tuning a local IT5 Small Language Model (SLM) on a dataset of Italian annotated clinical notes.
• Comparing the IT5 SLM's performance against general-purpose (GPT-4o mini) and medical-specific (MedGemma) models.
• Evaluating model performance based on sensitivity for rare disease marker extraction.

Main Results:

• The fine-tuned local IT5 SLM demonstrated competitive performance compared to GPT-4o mini and MedGemma for the specific task.
• The model achieved high sensitivity, which is critical for detecting rare conditions like Autoimmune Atrophic Gastritis (AAG).

Conclusions:

• Task-specific, locally deployed SLMs are a viable and advantageous solution for privacy-preserving clinical applications.
• Local SLMs can effectively process sensitive health data on-device, mitigating risks associated with cloud-based LLMs.