Comparative Analysis of Temporal Difference Learning Methods to Learn General Value Functions of Lower-Limb Signals.

Area of Science:

• Biomedical Engineering
• Robotics
• Machine Learning

Background:

• Millions face paralysis, impacting motor function and mobility.
• Current exoskeletons lack real-time adaptation to user biomechanics and environments.
• Reinforcement learning can enhance exoskeleton effectiveness for rehabilitation.

Purpose of the Study:

• To evaluate temporal difference (TD) learning methods for predicting lower-limb sensor signals.
• To compare the speed and accuracy of TD($\lambda$), TOTD, and SwiftTD algorithms.
• To inform the selection of predictive algorithms for adaptive exoskeletons.

Main Methods:

• Utilized temporal difference learning algorithms: TD($\lambda$), TOTD, and SwiftTD.
• Predicted sensor data including electromyography (muscle activation), underfoot pressure, and joint angles.
• Assessed algorithm performance based on convergence speed and error rates.

Main Results:

• SwiftTD demonstrated faster convergence across various sensor signals.
• TOTD generally achieved lower convergence errors compared to other methods.
• Algorithm performance varied depending on the specific signal being predicted.

Conclusions:

• The choice of TD learning algorithm impacts prediction accuracy and speed for exoskeleton control.
• Informed algorithm selection is crucial for developing adaptive, machine learning-controlled assistive devices.
• Optimized predictive algorithms will enhance exoskeleton performance, improving mobility for individuals with paralysis.