Adaptive penalty method with adam optimizer for enhanced convergence in optical waveguide mode solvers

We propose a cutting-edge penalty method for optical waveguide mode solvers, integrating the Adam optimizer into pseudospectral frequency-domain (PSFD) frameworks. This strategy enables adaptable boundary fluctuations at material interfaces, significantly enhancing numerical convergence and stability. The Adam optimizer, an adaptive algorithm, is deployed to determine the penalty coefficient, greatly improving convergence rates and robustness while effectively incorporating boundary conditions into the interfaces of subdomains. Our solver evaluates the numerical performance of optical waveguides by calculating effective indices of standard benchmark waveguides with high accuracy. This method diminishes numerical boundary errors and provides a marked increase in convergence speed and superior accuracy when compared to conventional methods and even metaheuristic optimization methods, all while maintaining the inherent global spectral accuracy of the PSFD.