A Novel Deep Reinforcement Learning Approach for Dynamic Proportional-Integral Control in Scanning Probe Microscopy

Area of Science:

• Materials Science
• Physics
• Computer Science

Background:

• Scanning Probe Microscopy (SPM) faces challenges with nonlinear, time-varying materials and abrupt topographical changes, causing instability and artifacts.
• Traditional proportional-integral (P-I) controllers lack adaptability to dynamic conditions in SPM.
• These limitations hinder high-resolution imaging of complex samples.

Purpose of the Study:

• To develop an adaptive control system for SPM that overcomes limitations of fixed-parameter controllers.
• To enhance the stability and reduce artifacts in SPM imaging of challenging materials.
• To leverage deep reinforcement learning (DRL) for real-time control strategy optimization.

Main Methods:

• Introduction of the Parallel Integrated Control and Training System (PICTS) utilizing deep reinforcement learning (DRL).
• Real-time dynamic adjustment of control strategies to stabilize probe-sample interactions.
• Implementation of a field-programmable gate array (FPGA) for efficient critical task processing.

Main Results:

• The DRL-based controller reduced deflection errors by 26%-90% compared to commercial fixed-parameter controllers.
• Achieved more stable SPM images with significantly fewer artifacts.
• Statistical analysis confirmed improved precision, with error values concentrated near zero.

Conclusions:

• The PICTS system effectively enhances SPM imaging quality and stability, particularly for samples with sharp edges, soft multiphase materials, or complex topographies.
• The integration of DRL and FPGA offers an efficient solution without requiring high-performance computing.
• This approach paves the way for advanced SPM applications in demanding research environments.