Low complexity deep neural network equalizer based on the multi-source domain transfer learning in IMDD system.

Area of Science:

• Optical Communications
• Machine Learning
• Signal Processing

Background:

• Deep neural network (DNN) equalizers are crucial for mitigating signal impairments in intensity-modulation and direct-detection (IMDD) systems.
• Training DNN-based equalizers can be computationally expensive and data-intensive, limiting their practical application.
• Existing transfer learning methods may not fully address the generalization and stability challenges in diverse channel conditions.

Purpose of the Study:

• To develop a novel multi-source domain transfer learning (MST) scheme to reduce the training cost of DNN-based equalizers for IMDD systems.
• To improve the generalization ability and stability of DNN equalizers across various channel parameters.
• To validate the effectiveness of the proposed MST equalizer in a practical high-speed IMDD system.

Main Methods:

• Designed a multi-source domain transfer learning (MST) scheme utilizing data from different channel parameters.
• Constructed a multi-source domain dataset by proportionally selecting data with varying channel characteristics.
• Trained the source domain within a single task to enhance model generalization and stability.

Main Results:

• The proposed MST equalizer demonstrated effectiveness in an 80Gb/s PAM-4 IMDD short-reach system.
• Achieved a bit error rate meeting the hard decision-forward error correction threshold.
• Reduced iteration epochs by 87% and training data by 65% compared to conventional DNN equalizers.

Conclusions:

• The MST scheme offers a significant reduction in training cost for DNN-based equalizers in IMDD systems.
• The proposed method ensures model generalization and stability, crucial for real-world optical communication systems.
• MST provides a more efficient and practical approach to deploying advanced equalization techniques.