Learning Actions From Natural Language Instructions Using an ON-World Embodied Cognitive Architecture.

Area of Science:

• Robotics
• Artificial Intelligence
• Cognitive Science

Background:

• Robots require human-like perception and natural language understanding for real-world deployment.
• Linking abstract language to physical actions is a significant challenge in artificial agents.

Purpose of the Study:

• To propose a novel brain-inspired architecture for mapping language to perception and motor representations in humanoid robots.
• To enable robots to learn novel semantic meanings and adapt to new environments through interactive, open-ended learning.

Main Methods:

• Implementing a cognitive architecture based on Baddeley's Working Memory (WM) model for scalable knowledge representation.
• Combining human spoken utterances with the robot's internal knowledge map for task accomplishment.
• Utilizing an interactive learning method for flexible run-time acquisition of linguistic forms and real-world information.

Main Results:

• Demonstrated robust bi-directional linking of language with the physical environment in humanoid robots.
• Showcased the ability to solve various manipulation tasks with limited initial knowledge.
• Verified successful gradual learning from run-time interaction with a tutor.

Conclusions:

• The proposed methodology effectively integrates language and perception for humanoid robots.
• The brain-inspired architecture supports incremental, open-ended learning and adaptation to novel situations.
• Robots can achieve task goals by learning abstract linguistic concepts through interaction.