Panoptic-PartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Panoptic Part Segmentation (PPS) unifies panoptic and part segmentation.
• Existing methods lack shared computation and task association for things, stuff, and parts.
• Previous metrics like PartPQ are biased towards panoptic quality (PQ).

Purpose of the Study:

• To develop the first end-to-end unified framework for Panoptic Part Segmentation (PPS).
• To introduce a new metric, Part-Whole Quality (PWQ), for more accurate task evaluation.
• To improve part segmentation quality through enhanced architectural designs.

Main Methods:

• Designed Panoptic-PartFormer, a meta-architecture decoupling part and thing/stuff features using object queries.
• Proposed Part-Whole Quality (PWQ) metric to decouple part and panoptic segmentation errors.
• Introduced Panoptic-PartFormer++ with a part-whole cross-attention scheme for improved part segmentation.

Main Results:

• Panoptic-PartFormer++ achieved state-of-the-art results on Cityscapes and Pascal Context PPS datasets.
• Improvements of 2% PartPQ and 3% PWQ on Cityscapes, and 5% PartPQ on Pascal Context were observed.
• The proposed PWQ metric offers a better evaluation perspective for PPS.

Conclusions:

• Panoptic-PartFormer and Panoptic-PartFormer++ demonstrate the effectiveness of unified architectural designs for PPS.
• The new models serve as strong baselines for future research in unified segmentation tasks.
• The proposed PWQ metric provides a more robust evaluation for panoptic part segmentation.