Open-Set Anomaly Segmentation in Complex Scenarios

Area of Science:

• Computer Vision
• Machine Learning
• Autonomous Systems

Background:

• Accurate segmentation of out-of-distribution (OoD) objects, or anomalies, is critical for safety-critical applications like autonomous driving.
• Existing benchmarks for anomalous segmentation primarily evaluate performance under ideal weather conditions, failing to address real-world challenges such as low illumination, fog, and heavy rain.
• This oversight leads to untrustworthy evaluations and highlights safety risks in open-set environments.

Purpose of the Study:

• To introduce ComSAmy, a novel benchmark for complex scenario anomaly segmentation, designed to evaluate model performance under diverse adverse weather and driving conditions.
• To assess the limitations of current state-of-the-art anomalous segmentation models in realistic, challenging open-world scenarios.
• To propose a new method, Energy-Entropy Learning (EEL) and a diffusion-based data synthesizer, to enhance anomaly segmentation robustness.

Main Methods:

• Development of the ComSAmy benchmark, featuring a wide range of adverse weather, dynamic environments, and anomaly types.
• Evaluation of existing anomalous segmentation models on the ComSAmy benchmark to identify performance gaps.
• Proposal of a novel Energy-Entropy Learning (EEL) strategy to leverage complementary energy and entropy information.
• Introduction of a diffusion-based synthesizer for generating diverse and high-quality anomalous training data.

Main Results:

• State-of-the-art anomalous segmentation models exhibit significant deficiencies in complex scenarios, posing safety risks.
• The proposed diffusion-based synthesizer effectively generates diverse and high-quality anomalous images.
• The integrated DiffEEL framework significantly enhances existing models, demonstrating effectiveness and generalizability.
• Average improvements of 4.96% in AUPRC and 9.87% in FPR95 were observed on public and ComSAmy benchmarks.

Conclusions:

• Current anomalous segmentation methods are insufficient for safe deployment in real-world, adverse conditions.
• The ComSAmy benchmark provides a more realistic evaluation of model performance.
• The proposed DiffEEL framework offers a robust and generalizable solution for improving anomaly segmentation in complex environments.
• The developed methods contribute to safer and more reliable autonomous driving systems.