Dynamic noise-masked 3D constellation encryption scheme based on gated recurrent unit networks

We propose a dynamic noise-masked three-dimensional constellation encryption scheme based on a gated recurrent unit (GRU) neural network. A three-dimensional six-row stacking (SRS) structure is employed for constellation shaping to enhance spectral efficiency and transmission performance. The transmitted data are used to dynamically generate initial keys for a chaotic model via GRU training, enabling key evolution synchronized with data changes. This ensures that the system operates under continuous dynamic encryption. Noise-masking technology is further introduced to embed dynamic keys into encrypted data, ensuring secure transmission and accurate key recovery at the receiver. Experimental validation is conducted through four transmissions of three groups of 16QAM signals at 5.12 Gb/s over 1.5 km of hollow-core fiber. The results show that the proposed 3D-SRS constellation achieves a 0.31 dB performance gain over the traditional cubic (TC) structure at the forward error correction (FEC) threshold. A unique key is assigned to each transmission, while the chaotic system provides an extensive key space of 10¹³⁴, substantially improving resistance to brute-force attacks. Meanwhile, the dissimilarity index (DI) matrix shows that the generation of dynamic keys results in variations of at least the order of 10^-2, effectively eliminating the possibility of key reuse. These results demonstrate that the proposed scheme effectively improves transmission security and supports high-reliability, high-capacity encrypted optical communications.