Fed-XLM-R: a privacy-preserving federated framework with adapter-scoped differential privacy for mental health...

Mental health triage-level intent classification in linguistically diverse and resource-constrained clinical environments requires models that simultaneously meet strict patient data privacy regulations, cross-lingual generalization to low-resource languages, and real-time inference on edge hardware. No existing federated NLP system has jointly demonstrated this combination. This work proposes Fed-XLM-R, a unified federated learning framework designed to close that gap.

Fed-XLM-R integrates XLM-RoBERTa with lightweight bottleneck adapter layers, adapter-scoped differentially private stochastic gradient descent (DP-SGD), proximal regularization for straggler-robust aggregation (FedProx), and INT8 post-training quantization for edge deployment. Gradient perturbation is confined exclusively to the 0.87% trainable adapter subspace rather than the full 270-million-parameter model. The framework was evaluated in a simulated 10-client federation on a public benchmark, with intent classification trained and evaluated on English data. A cross-lingual diagnostic was additionally run on the XNLI benchmark across five languages (English, Spanish, Hindi, Swahili, Arabic) against an mBERT baseline, and canary phrase memorization analysis was conducted across all 50 federated rounds to assess data extraction risk.

At the clinically meaningful privacy budget of ε = 1.0 (δ = 10^-5, Fed-XLM-R achieves 96% accuracy retention, yielding an overall accuracy of 0.923, F1-score of 0.918, and AUC-ROC of 0.956, within 0.5% of a centralized baseline while providing formal differential privacy guarantees. Canary phrase analysis confirms the DP-SGD mechanism prevents training data extraction. The XLM-R backbone retains strong discriminative capacity in low-resource languages relative to mBERT on the XNLI diagnostic. Adapter-only parameter transmission reduces cumulative communication cost by 40.7-fold at the 85% accuracy threshold, at a per-round volume compatible with standard mobile data networks. INT8 quantization reduces inference latency from 1,974 ms to 523 ms on low-end CPU hardware, a 3.8-fold speedup.

These findings characterize the mechanism's behavior under controlled conditions in a simulated federation rather than demonstrating field deployment. The cross-lingual results support the feasibility of future multilingual extension but do not themselves constitute multilingual intent classification. The three-category label scheme (Anxiety/Depression/Normal) supports triage-level intent routing but has not been validated against standardized clinical instruments; the system is not proposed as a clinical diagnostic or screening tool. Overall, Fed-XLM-R demonstrates that privacy-preserving, communication-efficient, and edge-deployable federated NLP is achievable without substantial accuracy loss, offering a practical pathway toward privacy-compliant mental health triage systems pending real-world and multilingual validation.