Instance Motion Tendency Learning for Video Panoptic Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Video panoptic segmentation is crucial for understanding dynamic scenes.
• Existing methods struggle with instance association across frames and motion blur, leading to errors.

Purpose of the Study:

• To develop a novel network, the Instance Motion Tendency Network (IMTNet), for enhanced video panoptic segmentation.
• To address challenges in temporal coherence, instance association, and motion boundary ambiguity.

Main Methods:

• Introduced the Instance Motion Tendency Network (IMTNet) incorporating a Global Motion Tendency Module (GMTM) and a Motion Boundary Refinement Module (MBRM).
• GMTM leverages optical flows for robust instance association across frames.
• MBRM employs a hierarchical classifier to refine segmentation boundaries of moving objects.

Main Results:

• IMTNet demonstrated superior performance in video panoptic segmentation tasks.
• The proposed methods effectively reduced identity switches and improved handling of motion blur.
• Experiments on Cityscapes and Cityscapes-VPS datasets confirmed the network's effectiveness.

Conclusions:

• IMTNet offers a simple yet effective solution for video panoptic segmentation.
• The network successfully tackles key challenges like temporal coherence and boundary ambiguity.
• IMTNet sets a new benchmark, outperforming existing state-of-the-art approaches.