Evaluation of Federated Learning in Phishing Email Detection

Area of Science:

• Cybersecurity
• Machine Learning
• Artificial Intelligence

Background:

• Current AI phishing detection relies on centralized data, raising privacy, trust, and legal concerns.
• Organizations hesitate to share emails due to sensitive information risks, hindering global AI model training.
• Federated learning (FL) presents a privacy-preserving, distributed alternative, particularly relevant for multi-organization collaborations.

Purpose of the Study:

• To investigate the effectiveness and efficacy of federated learning (FL) for phishing email detection.
• To evaluate FL performance using deep neural network models (RNN and BERT) in multi-organization settings.
• To analyze FL's scalability and performance under varying data distributions.

Main Methods:

• Implemented federated learning (FL) with Recurrent Neural Network (RNN) and Bidirectional Encoder Representations from Transformers (BERT) models.
• Assessed FL performance across balanced and asymmetrical data distributions among collaborating organizations.
• Evaluated FL scalability by varying the number of organizations and the total email dataset size.

Main Results:

• FL demonstrated comparable performance to centralized learning for balanced datasets and few organizations.
• Increasing organizations with a fixed dataset decreased RNN accuracy but improved BERT accuracy slightly.
• Expanding the dataset by adding organizations improved FL convergence speed and organizational-level performance.
• Highly asymmetric data distributions led to unstable FL global model outputs and performance degradation.

Conclusions:

• Federated learning is a viable and privacy-preserving approach for phishing email detection in collaborative environments.
• FL performance is sensitive to data distribution asymmetry and the number of participating organizations.
• Further research is needed to optimize FL for robust phishing detection across diverse organizational data.