Performance vs. complexity in NN pre-distortion for a nonlinear channel.

Area of Science:

• Optical Communications
• Signal Processing
• Machine Learning

Background:

• High-bandwidth optical communication systems rely on digital-to-analog converters (DACs).
• DACs introduce quantization and bandlimited impairments, degrading signal quality.
• Digital pre-distortion (DPD) is crucial for mitigating these impairments.

Purpose of the Study:

• To propose and experimentally validate a neural network (NN) based digital pre-distortion (DPD) technique.
• To mitigate quantization and bandlimited impairments introduced by DACs in high-speed optical systems.
• To compare the performance of NN-DPD against traditional DPD methods.

Main Methods:

• Experimental validation using a 64 Gbaud 8-level pulse amplitude modulation (PAM-8) signal.
• Training of NN-DPD using both direct and indirect learning methods.
• Comparison with Volterra, look-up table (LUT), and linear DPD solutions.

Main Results:

• The proposed NN-DPD, trained via direct learning, outperforms Volterra, LUT, and linear DPDs by 0.9 dB, 1.9 dB, and 2.9 dB, respectively.
• An indirect learning recurrent NN offers comparable performance to Volterra at similar complexity.
• A direct learning recursive NN achieves superior performance beyond Volterra's capabilities.

Conclusions:

• NN-DPD is an effective method for mitigating DAC impairments in high-speed optical communications.
• Direct learning recursive NN-DPD provides state-of-the-art performance.
• Indirect learning recurrent NN-DPD offers a competitive alternative with balanced complexity and performance.