From text to motion: grounding GPT-4 in a humanoid robot "Alter3"

Area of Science:

• Robotics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Large Language Models (LLMs) face challenges in direct robot control due to hardware complexities.
• Integrating LLMs with robotic systems requires translating abstract language into precise physical actions.

Purpose of the Study:

• To introduce Alter3, a humanoid robot capable of spontaneous motion generation.
• To demonstrate the effective application of a Large Language Model (LLM) for direct robot control.
• To explore the robot's ability to adapt to verbal feedback for action refinement.

Main Methods:

• Integration of GPT-4, a sophisticated LLM, with the Alter3 humanoid robot.
• Development of a system to translate linguistic descriptions of actions into robot-executable programs.
• Utilizing few-shot learning for rapid adaptation to new tasks and verbal feedback.

Main Results:

• Alter3 successfully generated spontaneous motions based on textual prompts, performing diverse actions like adopting a "selfie" pose.
• The robot demonstrated adaptability to verbal feedback for pose adjustments without manual reprogramming.
• The system showcased effective few-shot learning capabilities in a robotic context.

Conclusions:

• The integration of LLMs like GPT-4 with humanoid robots offers a viable pathway for spontaneous motion generation.
• This approach bridges the gap between linguistic understanding and physical embodiment in robotics.
• The research opens new possibilities for creating more adaptive and interactive humanoid robots.