Unidirectional pruned gated recurrent unit network for fiber nonlinearity mitigation in optical communication systems

Area of Science:

• Optical Communications
• Nonlinear Optics
• Machine Learning for Signal Processing

Background:

• Fiber nonlinear distortion limits optical transmission capacity.
• Existing deep learning methods like Bi-LSTM and Bi-GRU offer impairment mitigation but have high computational costs.
• Bidirectional networks introduce dual-pass iterations, increasing complexity.

Purpose of the Study:

• To develop a computationally efficient deep learning model for mitigating fiber nonlinear distortion.
• To reduce the complexity of GRU-based nonlinear compensation.
• To improve the practicality of deep learning for optical transmission systems.

Main Methods:

• Inspired by first-order perturbation theory, a direct link between GRU architecture and perturbative nonlinear compensation was established.
• The reset gate in GRU was pruned, reducing internal complexity by 33% without performance loss.
• A unidirectional propagation structure was adopted, eliminating dual-pass iterations and reducing complexity by an additional 50%.

Main Results:

• The proposed unidirectional pruned GRU (Uni-PGRU) achieved a 0.60 dB Q-factor improvement over linear compensation in a 1600 km transmission system.
• Uni-PGRU performance matched Bi-GRU and surpassed digital backpropagation (4 steps/span).
• Uni-PGRU demonstrated significantly lower computational complexity (1.9% of Bi-GRU) compared to other methods at equivalent performance.

Conclusions:

• The Uni-PGRU offers a highly efficient and practical solution for mitigating fiber nonlinearity.
• Pruning the reset gate and adopting a unidirectional structure are effective strategies for complexity reduction.
• This work advances the deployment of deep learning in high-capacity optical communication systems.