A large language model-based autonomous agent for scanning tunnelling microscopy
Related Experiment Videos
Juan Xiang1, Zhipeng Xiang1, Zhiwen Zhu1
1Materials Genome Institute, Shanghai University 200444 Shanghai China qiangsun@shu.edu.cn.
Abstract:
Scanning tunnelling microscopy (STM) provides real space access to molecular and atomic structures, but autonomous operation remains constrained by the need for expert decisions in parameter optimization, image assessment, data interpretation and probe recovery. Most autonomous scanning probe microscopy systems remain model-centric and task-specific, relying on labelled training data, fixed output classes, predefined reward functions or specific protocols. Here, we introduce STMClaw, an LLM-based autonomous STM agent implemented as a modular workflow. It coordinates autonomous STM experiments by integrating natural language instructions, instrument feedback, and retrieved domain knowledge, without requiring additional task-specific model training. The agent comprises modules for trajectory planning, image evaluation, probe conditioning, and molecular assignment based on literature. In an autonomous run, STMClaw generated 606 records, 554 positions were scanned and evaluated by the LLM and 52 candidate positions were skipped. The evaluator achieved F1 scores of 0.86 and 0.90 for the good and bad image classes using 13 reference examples per class, respectively. It also enabled the recovery of molecularly resolved images after probe degradation and linked STM images to evidence retrieved from the literature. These results support the feasibility of an LLM-based autonomous STM agent for adaptive, knowledge-aware surface science experimentation.
Related Concept Videos
Overview of Microscopy Techniques
Atomic Force Microscopy
The AFM Probe
The probe is regarded as the heart of any AFM setup and comprises the...