Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification.

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Natural Language Processing

Background:

• Large Language Models (LLMs) are increasingly used in healthcare for tasks like disease diagnosis and abnormality classification from medical reports.
• Fine-tuning LLMs on private patient data enhances accuracy but poses significant privacy risks due to potential data memorization and extraction.
• Privacy-preserving methods for fine-tuning LLMs in medical text classification, particularly for multi-abnormality classification, are underexplored.

Purpose of the Study:

• To propose and evaluate a novel differentially private (DP) fine-tuning approach for LLMs to perform multi-abnormality classification on text-based radiology reports.
• To address the critical need for privacy preservation in applying LLMs to sensitive medical data.
• To investigate the privacy-utility trade-off in DP fine-tuning for medical text classification.

Main Methods:

• Developed a differentially private (DP) fine-tuning framework using Low Rank Adaptation (LoRA) for LLMs.
• Leveraged DP optimization techniques to fine-tune LLMs on local patient data, mitigating risks of sensitive information leakage.
• Utilized labels generated by a larger LLM to fine-tune a smaller LLM for accelerated inference while adhering to privacy constraints.

Main Results:

• The proposed DP-LoRA framework achieved high performance on the MIMIC-CXR dataset, with weighted F1-scores up to 0.89 under moderate privacy budgets ($ϵ=10$).
• Performance closely approached that of non-private LoRA (0.90) and full fine-tuning (0.96), indicating minimal utility loss.
• Demonstrated the efficacy of the DP fine-tuning method across varying privacy regimes and datasets (MIMIC-CXR, CT-RATE).

Conclusions:

• The DP-LoRA approach effectively enables privacy-preserving multi-abnormality classification from radiology reports using LLMs.
• Strong privacy protection can be achieved with only moderate performance degradation, making LLMs more viable for sensitive healthcare applications.
• This work represents the first study to incorporate DP fine-tuning of LLMs for multi-abnormality classification in text-based radiology reports.