Visual Rewards From Observation for Sequential Tasks: Autonomous Pile Loading.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Designing reward functions for reinforcement learning (RL) in real-world robotics is a significant challenge.
• Field robotics faces additional hurdles like limited data, training time, and environmental variability.
• Existing reward learning techniques and visual representations in robotics are reviewed.

Purpose of the Study:

• To investigate the potential of a visual reward association approach for a real-scale field robotics application.
• To evaluate this approach in an autonomous pile loading task conducted outdoors across three seasons.
• To adapt and validate a cumulative reward framework combining process stage and task completion predictions.

Main Methods:

• Review of reward learning techniques and visual representations in robotics.
• Application of a visual observation-based reward association method to autonomous pile loading.
• Utilizing supervised classification for process stage prediction models.
• Employing task-specific contrastive features for terminal stage prediction.

Main Results:

• The study successfully adapted a visual reward approach for outdoor autonomous pile loading.
• Performance was evaluated across summer, autumn, and winter conditions.
• A cumulative reward framework integrating process stage and terminal stage predictions was implemented.

Conclusions:

• The visual reward association method shows potential for real-world field robotics applications.
• The framework effectively combines visual cues for reward shaping in dynamic environments.
• Seasonal variations did not prevent successful adaptation of the autonomous pile loading task.