Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation

Shyam Nandan Rai , Fabio Cermelli , Barbara Caputo

IEEE Transactions on Pattern Analysis and Machine Intelligence +

|

June 27, 2024

View abstract on PubMed

Summary

This study introduces Mask2Anomaly, a novel mask classification method for improved autonomous driving perception. It effectively detects unknown objects by shifting from pixel-level to mask-level analysis, enhancing safety and reliability.

Area Of Science

Computer Vision
Artificial Intelligence
Autonomous Systems

Background

Traditional per-pixel classification for anomaly segmentation in autonomous driving suffers from boundary uncertainty and false positives.
Lack of contextual semantics in per-pixel methods hinders accurate detection of unknown or anomalous objects.

Purpose Of The Study

To propose a paradigm shift from per-pixel classification to mask classification for anomaly segmentation.
To introduce Mask2Anomaly, a mask-based method for joint anomaly, open-set semantic, and open-set panoptic segmentation.
To enhance the detection of anomalies and unknown objects in autonomous driving scenarios.

Main Methods

Developed Mask2Anomaly, a mask-classification architecture.
Incorporated a global masked attention module for focused foreground/background analysis.
Utilized mask contrastive learning to differentiate anomalies from known classes.
Implemented a mask refinement solution to minimize false positives.
Introduced a novel approach for mining unknown instances based on mask properties.

Main Results

Mask2Anomaly demonstrates the feasibility of mask classification for autonomous driving perception tasks.
Achieved new state-of-the-art results on benchmarks for anomaly segmentation, open-set semantic segmentation, and open-set panoptic segmentation.
The method effectively reduces uncertainty around object boundaries and minimizes false positives.

Conclusions

Mask2Anomaly represents a significant advancement in segmenting unknown objects for autonomous driving.
The mask-based approach offers a more robust and accurate solution compared to traditional per-pixel methods.
This work paves the way for more reliable and safer autonomous driving systems through improved environmental perception.