Where Can We Help? A Visual Analytics Approach to Diagnosing and Improving Semantic Segmentation of Movable Objects

Area of Science:

• Computer Vision
• Artificial Intelligence
• Autonomous Systems

Background:

• Deep neural network (DNN) models are crucial for semantic segmentation in autonomous driving.
• Evaluating DNNs is challenging due to their black-box nature, especially for critical objects like pedestrians and lost cargo.
• Ensuring the accuracy and robustness of these models is paramount for safety in autonomous driving applications.

Purpose of the Study:

• To propose VASS, a Visual Analytics approach for diagnosing and enhancing semantic segmentation models.
• To specifically improve the accuracy and robustness of models for critical objects in diverse driving scenarios.
• To provide actionable insights for model improvement through visual analysis and adversarial testing.

Main Methods:

• Developed a context-aware spatial representation learning technique to extract object features (position, size, aspect ratio) relative to scene context.
• Utilized spatial representations for visual summarization to analyze model performance.
• Employed spatial representations to guide the generation of adversarial examples for evaluating spatial robustness.

Main Results:

• Demonstrated VASS's effectiveness through case studies on lost cargo and pedestrian detection.
• Showcased quantitative improvements in model performance.
• Provided actionable insights derived from VASS for enhancing model accuracy and robustness.

Conclusions:

• VASS offers a powerful approach to diagnose and improve semantic segmentation models for autonomous driving.
• The method effectively enhances model performance, particularly for critical object detection.
• Visual analytics combined with adversarial example generation provides valuable insights for robust AI systems.