Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge...

Area of Science:

• Robotics
• Machine Learning
• Control Systems

Background:

• Predictive knowledge enhances robot control and other applications.
• Online, incremental learning through environmental interaction is key for robotics.
• A challenge is selecting appropriate learning parameters, such as learning rates or step sizes.

Purpose of the Study:

• To examine online step-size adaptation using Temporal-Difference Incremental Delta-Bar-Delta (TIDBD).
• To evaluate TIDBD as a practical alternative to classic Temporal-Difference (TD) learning.
• To demonstrate TIDBD's ability to detect sensor failures in robotic applications.

Main Methods:

• Applied TIDBD to a Modular Prosthetic Limb, a sensor-rich robotic arm.
• TIDBD learns and adapts step sizes on a feature level, enabling simultaneous step-size tuning and representation learning.
• Compared TIDBD's performance to classic TD learning with extensive parameter searches.

Main Results:

• TIDBD performs comparably to hand-tuned TD learning in predicting robotic data streams.
• TIDBD automatically detects patterns indicative of sensor failures, a common issue in robotic applications.
• TIDBD demonstrates robustness to initial step-size values, outperforming classic TD.

Conclusions:

• TIDBD is a practical and robust method for online step-size adaptation in robotic learning.
• The approach improves the ability of robotic devices to learn from environmental interactions.
• These findings enhance capabilities for autonomous agents and robots.