Gaussian mixture model-hidden Markov model based nonlinear equalizer for optical fiber transmission.

Area of Science:

• Optical communications
• Signal processing
• Machine learning

Background:

• Increasing demand for high-speed data transmission drives data center development.
• Nonlinear effects in optical fiber systems become significant at higher transmission speeds.
• Conventional digital signal processing (DSP) algorithms struggle to accurately capture nonlinear distortions.

Purpose of the Study:

• To propose a novel nonlinear equalizer with reduced computational complexity.
• To leverage statistical signal characteristics for improved equalization performance.
• To evaluate the equalizer's effectiveness in mitigating nonlinear distortions in optical interconnects.

Main Methods:

• Development of a Gaussian mixture model (GMM)-hidden Markov model (HMM) based nonlinear equalizer.
• Utilization of received signal statistical characteristics as a priori information.
• Evaluation in a Pulse Amplitude Modulation with 4 levels (PAM-4) modulated Vertical-Cavity Surface-Emitting Laser-Multimode Fiber (VCSEL-MMF) optical interconnect link.

Main Results:

• The GMM-HMM based equalizer demonstrates excellent capability in mitigating nonlinear distortions.
• Achieved similar Bit Error Rate (BER) performance compared to Recurrent Neural Network (RNN) based methods.
• The proposed equalizer exhibits approximately 73% lower computational complexity than RNN-based methods.

Conclusions:

• The GMM-HMM based nonlinear equalizer offers a computationally efficient solution for high-speed optical data transmission.
• This method effectively reduces computational complexity while maintaining competitive BER performance.
• The approach shows promise for future high-speed optical interconnect systems facing nonlinear impairments.