PRBS orders required to train ANN equalizer for PAM signal without overfitting.

Area of Science:

• Optical Communications
• Signal Processing
• Machine Learning

Background:

• Artificial neural network (ANN)-based nonlinear equalizers (NLEs) are effective for intensity-modulation/direct-detection (IM/DD) systems.
• M-ary pulse amplitude modulation (PAM-M) is widely used in high-speed IM/DD systems.
• Training ANN-NLEs with pseudorandom binary sequences (PRBSs) can lead to overfitting if PRBS order is too low.

Purpose of the Study:

• To provide a guideline for selecting PRBSs to train ANN-NLEs for PAM-M signals.
• To prevent the overfitting problem during ANN-NLE training.
• To ensure optimal performance of trained ANN-NLEs on new input sequences.

Main Methods:

• Determining the minimum PRBS orders required for ANN-NLE training based on equalizer input size.
• Theoretical analysis of PRBS selection for ANN-NLE training.
• Computer simulations to validate the theoretical findings.

Main Results:

• A selection guideline for PRBSs to train ANN-NLEs in PAM-M IM/DD systems is established.
• The guideline ensures that the ANN-NLE is trained effectively without overfitting.
• The minimum required PRBS order is determined for a given equalizer input size.

Conclusions:

• The proposed PRBS selection guideline effectively prevents overfitting in ANN-NLEs for PAM-M IM/DD systems.
• This guideline ensures robust performance of the trained equalizer across different symbol coding schemes.
• The findings are crucial for optimizing the training process of ANN-NLEs in high-speed optical communication systems.