Res-FLNet: human-robot interaction and collaboration for multi-modal sensing robot autonomous driving tasks based on...

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Autonomous vehicles and robotics require advanced navigation and Human-Robot Interaction (HRI) capabilities.
• Multimodal sensing in robots (cameras, LiDAR, radar) raises significant privacy and data security concerns.
• Existing methods struggle to balance performance with the need for data privacy in autonomous systems.

Purpose of the Study:

• To introduce Res-FLNet, a novel framework for privacy-preserving autonomous driving using multimodal sensing.
• To leverage Federated Learning (FL) for secure, decentralized model training on individual robots.
• To enhance Human-Robot Interaction and Collaboration through privacy-preserving knowledge sharing.

Main Methods:

• Res-FLNet integrates ResNet-50 for visual feature extraction and Long Short-Term Memory (LSTM) for sequential data analysis.
• Federated Learning (FL) is employed to train models locally on robots, preventing raw data sharing.
• Model updates are aggregated centrally to build collective intelligence while maintaining individual data privacy.

Main Results:

• Res-FLNet demonstrated superior accuracy, robustness, and privacy preservation on KITTI, Waymo, ApolloScape, and BDD100K datasets.
• The framework achieved state-of-the-art performance in autonomous driving tasks.
• Experiments confirmed Res-FLNet's adaptability and generalization capabilities across diverse driving scenarios.

Conclusions:

• Res-FLNet offers an effective solution for privacy-preserving multimodal autonomous driving.
• The integration of FL enables secure knowledge sharing, crucial for advancing HRI and collaboration.
• Res-FLNet shows significant potential for deployment in complex, dynamic environments for multi-modal sensing robots.