Sensory-motor control with large language models via iterative policy refinement.

Area of Science:

• Artificial Intelligence
• Robotics
• Reinforcement Learning

Background:

• Embodied agents require sophisticated control policies for real-world interaction.
• Large language models (LLMs) possess advanced reasoning capabilities but lack direct physical control mechanisms.

Purpose of the Study:

• To develop a method enabling LLMs to generate and refine control policies for embodied agents.
• To bridge the gap between symbolic reasoning of LLMs and sub-symbolic sensory-motor data for agent control.

Main Methods:

• LLMs generate an initial control strategy based on textual descriptions of agent, environment, and goals.
• Iterative refinement of the control strategy using performance feedback and sensory-motor data from agent interaction.
• Validation on classic control tasks (Gymnasium) and physics-based tasks (MuJoCo inverted pendulum).

Main Results:

• The proposed method effectively enables LLMs to control embodied agents.
• Successful application with compact LLMs like GPT-oss:120b and Qwen2.5:72b.
• Optimal or near-optimal solutions were achieved by integrating symbolic reasoning with real-time interaction data.

Conclusions:

• LLM-driven control policies offer a promising approach for embodied agent navigation and task execution.
• The iterative refinement process enhances policy robustness and adaptability.
• This method facilitates the integration of LLM's general intelligence with physical world interactions.