Multimodal Shared Autonomy for Heavy-Load UAV Operations with Physics-Aware Cooperative Control.

Area of Science:

• Robotics
• Artificial Intelligence
• Aerospace Engineering

Background:

• Heavy-load UAVs face operational challenges due to complex dynamics and unpredictable environments.
• Current control methods like manual teleoperation or full autonomy have limitations in cognitive load and reliability.

Purpose of the Study:

• To develop an advanced shared autonomy framework, the Multimodal Fusion Cooperation Network (MFCN).
• To integrate diverse operator inputs (speech, gestures, haptics) for intuitive and effective UAV control.

Main Methods:

• The MFCN employs cross-modal feature fusion to interpret real-time operator intent.
• A cooperative control policy with physics-aware constraints translates intent into stable flight commands.
• Framework validated through extensive semi-physical simulations and real-world experiments.

Main Results:

• Significant improvements in task success rate, positioning accuracy, and payload stability were observed.
• Reduced task completion time and operator cognitive workload compared to baseline methods.
• Demonstrated superior performance over manual, unimodal, and heuristic multimodal approaches.

Conclusions:

• The MFCN offers a robust solution for enhancing heavy-load UAV operations.
• Shared autonomy integrating multimodal inputs improves control effectiveness and operator experience.
• The framework shows promise for safe and efficient deployment in complex missions.