Privacy-Preserving Large Language Model for Matching Findings and Tracking Interval Changes in Longitudinal Radiology...

Area of Science:

• Artificial Intelligence in Radiology
• Natural Language Processing for Medical Reports

Background:

• Radiologists manually compare prior and current imaging reports to assess changes.
• Large Language Models (LLMs) show potential for identifying report findings but require validation for tracking interval changes.

Purpose of the Study:

• To evaluate the utility of a privacy-preserving LLM for matching findings between prior and follow-up radiology reports.
• To assess the LLM's capability in tracking interval changes in lesion size.

Main Methods:

• A two-stage framework using LLMs to match findings and predict interval change status (increase, decrease, stable).
• Evaluation on internal body MRI and external non-contrast chest CT datasets, with radiologist agreement measured by Cohen's Kappa (κ).

Main Results:

• TenyxChat-7B LLM achieved an 85.4% F1-score for finding matching on the internal dataset.
• Moderate agreement (κ=0.46) for interval change detection on internal data; substantial agreement (κ=0.64) on external data.
• The LLM showed strong performance in matching findings (81.8% F1-score) and interval changes (77.4%) on the external dataset.

Conclusions:

• The TenyxChat-7B LLM effectively matches longitudinal radiology report findings and tracks interval changes with moderate to substantial agreement.
• LLMs can enhance structured reporting by pre-filling findings sections and improving communication between radiologists and referring physicians.