A Federated Benchmark for Clinical Natural Language Processing (FedDRAGON)

Area of Science:

• Clinical Natural Language Processing
• Federated Learning
• Artificial Intelligence in Healthcare

Background:

• Clinical natural language processing (NLP) often requires large, centralized datasets.
• Data privacy concerns and logistical challenges limit the centralization of clinical data.
• Federated learning (FL) offers a privacy-preserving alternative for training models on decentralized data.

Purpose of the Study:

• Introduce the FedDRAGON challenge, a novel benchmark for federated learning in clinical NLP.
• Evaluate the performance of federated learning models on information extraction tasks using real-world clinical data.
• Provide a publicly available resource for advancing research in decentralized clinical NLP.

Main Methods:

• Developed a federated learning benchmark (FedDRAGON) comprising 12 information extraction tasks.
• Utilized de-identified clinical reports from 4 Dutch healthcare centers.
• Trained and evaluated federated learning models against single-center and centralized baselines.

Main Results:

• Federated models demonstrated performance superior to single-center models.
• The performance of federated models approached that of centralized models.
• Established baseline results for federated learning on clinical NLP tasks.

Conclusions:

• Federated learning is a viable and effective approach for clinical NLP, even with decentralized data.
• The FedDRAGON benchmark facilitates reproducible research and development in privacy-preserving clinical AI.
• Public availability of the benchmark, code, and pre-trained models accelerates progress in the field.