Constellation size for probabilistic shaping under the constraint of limited ADC resolution

We investigate the optimal constellation size for probabilistic shaping (PS) under the constraint of different effective numbers of bits (ENOB) of the analog-to-digital converter (ADC) at the receiver. For a fixed entropy, increasing the constellation size brings a larger shaping gain, while it also leads to a higher peak-to-average-power ratio, which makes the signal more sensitive to the ENOB and fiber nonlinearity. Our experimental results show that PS-44 quadrature amplitude modulation (QAM) outperforms both PS-64 QAM and uniformly shaped 32 QAM (U-32 QAM) when the ENOB of ADC is lower than 4.5. The optimal launched power and the maximum achievable distance for U-32 QAM, PS-44 QAM, and PS-64 QAM are analyzed by simulation.