A platform for investigating prompt framing as interface parameters in foundation models for robotics

Area of Science:

• Robotics and Artificial Intelligence
• Human-Computer Interaction
• Machine Learning

Background:

• Foundation models, particularly large language models (LLMs), are increasingly used for robotic control, decision-making, and execution.
• Hybrid paradigms combining reinforcement learning (RL) agents with LLMs show promise for robotic control.
• The interface between RL agents and LLMs presents an opportunity to study prompt framing effects.

Purpose of the Study:

• To develop a controlled experimental platform for measuring the impact of prompt framing on hybrid LLM + RL systems.
• To understand how manipulating the interface between RL agents and LLMs affects a hybrid advisor-arbiter architecture's behavior.
• To evaluate the influence of different prompt framings on multi-step decision-making and control tasks.

Main Methods:

• Comparison of three agents: RL-only (tabular Q-learning), LLM-only (stateless action selection), and a hybrid LLM + RL agent.
• Evaluation in a simulated navigation environment under a constrained interaction budget (10 episodes per world).
• Ablation studies on advisor channels (random and null recommendations) and analysis of various prompt framings (navigation-role, narrative, relational personas).

Main Results:

• The hybrid LLM + RL agent outperformed both RL-only and LLM-only baselines in mean success and cumulative reward.
• Advisor-channel ablations decreased performance, indicating the value of structured advice.
• Different prompt framings (e.g., caregiver persona variants) produced heterogeneous effects on agent behavior.

Conclusions:

• Hybrid LLM + RL architectures can achieve superior performance in robotic control tasks compared to standalone RL or LLM agents.
• The structure and framing of prompts significantly influence the decision-making and control capabilities of hybrid AI systems.
• This work provides a valuable testbed and evaluation methodology for future research into prompt engineering for complex AI tasks.