DeepInteraction++: Multi-Modality Interaction for Autonomous Driving.

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Current autonomous driving systems often use multi-modal fusion, which can limit performance by not fully utilizing individual sensor data.
• Overlooking modality-specific strengths in fusion strategies hinders reliable scene understanding and perception.

Purpose of the Study:

• To propose a novel modality interaction strategy that preserves and exploits unique characteristics of individual sensor representations.
• To develop a framework, DeepInteraction++, that enhances autonomous driving perception by enabling effective inter-modality information exchange.

Main Methods:

• Introduced a DeepInteraction++ framework with a dual-stream Transformer encoder for modality-specific representation learning and integration.
• Incorporated object-centric feature alignment and global information spreading for robust perception.
• Designed a predictive interaction decoder for iterative refinement of predictions through modality-agnostic aggregation.

Main Results:

• The proposed framework demonstrated superior performance in 3D object detection tasks.
• Significant improvements were observed in end-to-end autonomous driving evaluations.
• The modality interaction strategy effectively exploited individual sensor strengths for enhanced scene understanding.

Conclusions:

• The novel modality interaction strategy overcomes limitations of traditional fusion methods in autonomous driving.
• DeepInteraction++ offers a more effective approach to multi-modal perception for autonomous systems.
• The framework's ability to maintain and leverage modality-specific information is key to its enhanced performance.