Recovering Sensitive Medical Text in Federated Learning

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Cybersecurity

Background:

• Federated Learning (FL) enables collaborative model training without sharing raw data, crucial for sensitive Electronic Health Records (EHRs).
• Despite FL's distributed nature, shared gradients during training pose privacy risks.
• Transformer-based language models are increasingly used in healthcare but their vulnerability to data reconstruction attacks is not fully understood.

Purpose of the Study:

• To assess the privacy risks of transformer-based language models in a cross-silo Federated Learning setup using clinical text.
• To evaluate the effectiveness of gradient inversion attacks, specifically the Decepticons method, in reconstructing sensitive information from gradients.
• To analyze the impact of batch size and sequence length on the quality of reconstructed clinical text.

Main Methods:

• Simulated a cross-silo Federated Learning environment with diverse French clinical reports (genetic, anesthesia, birth records).
• Employed the Decepticons gradient inversion attack method to reconstruct text from shared gradients.
• Varied batch size and sequence length parameters to measure their effect on reconstruction accuracy.

Main Results:

• High accuracy in reconstructing clinical text from gradients was achieved by a malicious server.
• Token-level text recovery exceeded 95% with batch size 1 and remained above 60% for sequences up to 512 tokens.
• Reconstructed text contained identifiable patient information, including names, dates, and genetic markers, confirming significant privacy vulnerabilities.

Conclusions:

• Federated Learning alone is insufficient for safeguarding sensitive clinical text due to gradient leakage.
• Gradient inversion attacks pose a serious threat to patient privacy in FL healthcare applications.
• Integration of robust privacy-preserving techniques is essential before deploying FL models with clinical data.