AMaze: an intuitive benchmark generator for fast prototyping of generalizable agents

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Vision

Background:

• Traditional agent training uses simple, static environments, limiting generalization.
• Current benchmarks often rely on costly human design or random generation for environmental diversity.
• Agents need improved generalization for real-world applications.

Purpose of the Study:

• Introduce AMaze, a novel benchmark generator for embodied agents navigating complex mazes.
• Facilitate human interaction for creating feature-specific mazes and understanding agent strategies.
• Evaluate different training regimes for agent generalization.

Main Methods:

• Developed AMaze, a benchmark generator for visual maze navigation tasks.
• Trained embodied agents using one-shot, scaffolding, and interactive training regimes.
• Tested agent performance and generalization capabilities in a discrete maze environment.

Main Results:

• Interactive and scaffolding training regimes significantly improved agent generalization compared to direct training.
• Median performance gains ranged from 50% to 100% depending on training regime and metrics.
• Maximal generalization performance was achieved through interactive, human-in-the-loop training.

Conclusions:

• AMaze provides a controllable, human-interactive benchmark generator for embodied AI.
• Human-in-the-loop training is crucial for developing robust and generalizable agents.
• The AMaze benchmark facilitates research into agent generalization and strategy interpretability.