Test-Time Correction: An Online 3D Detection System via Visual Prompting

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• Autonomous driving systems rely on accurate 3D object detection for safety.
• Current 3D detectors are typically trained offline and lack adaptability during deployment.
• Test-time errors in perception systems pose significant risks to autonomous vehicle safety.

Purpose of the Study:

• To introduce Test-time Correction (TTC), an online system for rectifying 3D detection errors during inference.
• To enhance the safety and adaptability of deployed autonomous driving systems.
• To enable real-time error correction without the need for model retraining.

Main Methods:

• Developed an Online Adapter (OA) module, a prompt-driven query generator, integrated with existing 3D detectors.
• Utilized visual prompts derived from auxiliary feedback (e.g., 2D detection mismatches, road descriptions, user clicks) to guide correction.
• Implemented a visual prompt buffer to store information from risky objects for continuous correction across frames.

Main Results:

• TTC significantly improves instant error rectification compared to traditional frozen 3D detectors.
• The system demonstrates effectiveness even with limited labels, in zero-shot settings, and under adverse driving conditions.
• Achieved reliable, adaptive, and versatile driving autonomy through consistent detection of risky objects.

Conclusions:

• TTC offers a novel approach to post-deployment online rectification for autonomous driving perception systems.
• The proposed method enhances the robustness and safety of autonomous vehicles by enabling real-time adaptation.
• This work encourages further research into online correction mechanisms for improving the reliability of AI systems in dynamic environments.