Model-based reinforcement learning for ultrasound-driven autonomous microrobots.

Area of Science:

• Robotics
• Artificial Intelligence
• Biomedical Engineering

Background:

• Reinforcement learning (RL) offers autonomous control for microrobots but faces challenges like large data needs and poor generalizability.
• Ultrasound-actuated microrobots require rapid, precise control in complex, high-dimensional action spaces, exceeding human operator capabilities.

Purpose of the Study:

• To develop a sample-efficient, model-based reinforcement learning algorithm for autonomous control of ultrasound-driven microrobots.
• To enable microrobots to learn from limited data and adapt to new environments effectively.

Main Methods:

• Implemented model-based reinforcement learning (MBRL) utilizing recurrent imagined environments for training.
• Trained the AI model in a simulation environment before fine-tuning on physical tasks.
• Utilized image-based learning for data-scarce scenarios.

Main Results:

• Achieved 90% success rate in collision avoidance and channel navigation after one hour of fine-tuning.
• Demonstrated successful generalization to new environments, improving from 50% to over 90% with 30 minutes of additional training.
• Showcased real-time manipulation of microrobots in complex vasculatures under static and flow conditions.

Conclusions:

• Model-based reinforcement learning provides an effective solution for autonomous microrobot control in data-scarce environments.
• AI-driven microrobots demonstrate significant potential for revolutionizing biomedical applications through precise navigation and manipulation.