A Novel Deep Reinforcement Learning Approach for Dynamic Proportional-Integral Control in Scanning Probe Microscopy

Scanning Probe Microscopy (SPM) measurements often encounter challenges due to the nonlinear and time-varying behavior of materials, as well as abrupt changes in structure, properties, or topography. These issues can lead to system instability and imaging artifacts. Traditional proportional-integral (P-I) controllers with fixed parameters struggle to adapt to such conditions. To address this limitation, this study introduces the Parallel Integrated Control and Training System (PICTS), which leverages deep reinforcement learning (DRL) to dynamically adjust control strategies in real time. This approach stabilizes probe-sample interactions, even for samples with sharp edges, soft multiphase materials, or complex topographies. Experimental results demonstrate that the DRL-based controller reduces deflection errors by 26% - 90% compared to commercial fixed-parameter controllers, producing more stable images with fewer artifacts. Statistical analyses confirm improved precision, with error values concentrated near zero. Additionally, the system employs a field-programmable gate array (FPGA) for critical tasks, ensuring efficient operation without the need for high-performance computing. By integrating advanced machine learning, this work enhances imaging quality and stability in demanding environments, paving the way for further innovations in SPM.