Detecting New Lesions Using a Large Language Model: Applications in Real-World Multiple Sclerosis Datasets.

Area of Science:

• Artificial Intelligence in Medicine
• Neuroimaging Analysis
• Multiple Sclerosis Research

Background:

• Neuroimaging is crucial for detecting new inflammatory activity in multiple sclerosis (MS).
• Electronic health records (EHRs) contain valuable narrative MRI reports that are challenging to analyze for research.
• Automating the extraction of discrete data from these reports can significantly benefit MS research.

Purpose of the Study:

• To develop a large language model (LLM) prompt for classifying narrative MRI reports.
• To assess the prompt's accuracy and efficiency in identifying new T2-weighted lesions (newT2w) and contrast-enhancing lesions (CEL).
• To demonstrate the LLM's application in monitoring B-cell depleting therapy (BCDT) response in MS patients.

Main Methods:

• An institutional ecosystem integrated healthcare data with ChatGPT4 for analyzing MS MRI reports (2000-2022).
• A refined LLM prompt (msLesionprompt) was created to classify newT2w and CEL.
• Validation included efficiency assessment (time, cost), comparison with manual annotations, and analysis of BCDT treatment predictors.

Main Results:

• The msLesionprompt achieved high accuracy for newT2w (97%) and CEL (96.8%) detection.
• 14,888 reports were processed in 4.13 hours at a cost of $28, with 79% showing no new activity.
• BCDT demonstrated >97% suppression of new inflammatory activity post-rebaseline scan.
• Neighborhood poverty (Area Deprivation Index) emerged as a predictor for inflammatory activity.

Conclusions:

• LLM-driven extraction of discrete information from narrative imaging reports is feasible and efficient.
• This automated approach can enhance real-world analyses of MS disease progression and treatment effectiveness.
• The methodology holds potential for augmenting various research applications in MS.