OptiDistillNet: Learning nonlinear pulse propagation using the student-teacher model

Area of Science:

• Nonlinear Optics
• Computational Physics
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) are effective for modeling complex physical phenomena.
• Model compression is crucial for deploying deep learning models in real-world applications, particularly in resource-constrained environments.
• Knowledge distillation (KD) is a promising technique for compressing large neural networks into smaller, more efficient ones.

Purpose of the Study:

• To develop a compressed CNN model for learning pulse evolution in nonlinear fibers by solving the nonlinear Schrodinger equation (NLSE).
• To apply a knowledge distillation (KD) framework to compress a CNN, creating a student network named OptiDistillNet.
• To evaluate the performance of the compressed model in terms of accuracy, convergence speed, and parameter efficiency.

Main Methods:

• Solving the nonlinear Schrodinger equation (NLSE) to generate training data for pulse evolution.
• Implementing a knowledge distillation (KD) framework to train a smaller student network (OptiDistillNet) from a larger teacher network.
• Utilizing a deep convolutional neural network (CNN) architecture for learning the pulse evolution dynamics.

Main Results:

• The OptiDistillNet achieved a mean square error (MSE) close to the teacher model's MSE of 1.04*10^-5, even with a model size reduction of up to 91.2%.
• The compressed student network demonstrated better generalization, faster convergence, and required fewer trainable parameters compared to the original model.
• This represents the first application of KD techniques to nonlinear optics for model compression.

Conclusions:

• The proposed KD-based compression framework (OptiDistillNet) is highly effective for nonlinear optics applications.
• The simplified architecture, fast optimization, and improved accuracy make OptiDistillNet suitable for practical applications, such as optical coherent communication systems.
• This work paves the way for efficient deep learning deployment in complex nonlinear optical systems.