Unidirectional pruned gated recurrent unit network for fiber nonlinearity mitigation in optical communication systems

Fiber nonlinear distortion remains a fundamental bottleneck that constrains the capacity of optical transmission systems. By exploiting the temporal correlation of nonlinear effects, bidirectional long short-term memory (Bi-LSTM) networks have demonstrated notable potential for impairment mitigation. However, their multiple gating mechanisms incur substantial computational overhead, limiting practical deployment. Although bidirectional gated recurrent units (Bi-GRUs) reduce complexity through gate simplification, the presence of two gating operations still imposes a considerable burden. Here, inspired by first-order perturbation theory, we establish for the first time a direct connection between the GRU architecture and perturbative nonlinear compensation. Theoretical analysis reveals that the reset gate can be pruned without compromising the preservation of essential historical nonlinear information, thereby reducing the internal complexity of the GRU by 33% without performance degradation. Furthermore, by adopting a unidirectional propagation structure, the model eliminates the dual-pass iterations inherent to bidirectional networks, achieving an additional 50% reduction in computational complexity. We experimentally validate the proposed approach in a 21-channel, 60-GBaud polarization-division-multiplexed 16-QAM wavelength-division-multiplexing system over 1600 km of fiber transmission. The resulting unidirectional pruned GRU (Uni-PGRU) delivers a 0.60 dB Q-factor improvement over linear compensation, matching the performance of Bi-GRU and surpassing digital backpropagation with four steps per span. Crucially, at equivalent compensation performance, the Uni-PGRU requires only 1.9% of the computational complexity of Bi-GRU, 33.6% of Co-GRU, 9.1% of digital backpropagation, and 36.4% of learned digital backpropagation, establishing it as a highly efficient and practically viable solution for fiber nonlinearity mitigation.