Low-complexity clustered raised-cosine-optimized FIR time-domain digital back-propagation for coherent optical...

This paper presents a low-complexity time-domain digital back-propagation (TD-DBP) algorithm that synergistically combines two techniques: raised-cosine-optimized finite impulse response (RC-FIR) filtering and coefficient clustering, for efficient joint compensation of chromatic dispersion (CD) and nonlinear impairments in coherent optical systems. The proposed algorithm effectively resolves two critical limitations in existing digital nonlinear compensation schemes: (i) the accuracy degradation in FIR TD-DBP implementations caused by accumulated approximation errors and excessive out-of-band gain in the FIR filter, especially when employing fine step sizes; and (ii) the substantial computational overhead associated with frequency-domain DBP (FD-DBP) due to repeated fast Fourier transform operations. In the proposed clustered RC-FIR TD-DBP algorithm, an optimized RC window function is employed to enhance the tap coefficients and suppress the out-of-band spectral leakage in the FIR filter, thereby improving CD compensation accuracy. Furthermore, the k-means++ clustering algorithm is implemented to intelligently eliminate redundant filter coefficients, reducing computational complexity. The Q-factor performance and computational complexity of the proposed algorithm are comprehensively evaluated through numerical simulations and experimental demonstrations in coherent optical transmission systems. In a 160 Gb/s dual-polarization 16-quadrature amplitude modulation (16QAM) simulation over 9 × 100 km standard single-mode fiber (SSMF), clustered RC-FIR TD-DBP achieves 216 real multiplications per symbol (RMpS) complexity, 55.5% reduction compared to the non-clustered RC-FIR TD-DBP (486 RMpS) with only a minimal 0.35 dB Q-factor penalty, 33.3% lower than FD-DBP (324 RMpS) while maintaining comparable transmission quality (0.19 dB Q-factor penalty). Experimental validation in an 80 Gb/s single-polarization 16QAM system over 9 × 100 km SSMF demonstrates that clustered RC-FIR TD-DBP reduces complexity to 234 RMpS, 58.0% reduction compared to the non-clustered RC-FIR TD-DBP (522 RMpS) with only a minimal 0.16 dB Q-factor penalty, 27.7% lower than FD-DBP (324 RMpS) with a negligible 0.05 dB penalty at the 7% hard-decision forward error correction (HD-FEC) threshold. These results clearly demonstrate that our approach successfully overcomes the fundamental trade-off between computational efficiency and compensation accuracy in digital nonlinear equalization.