LOMM: Latest Object Memory Management for Temporally Consistent Video Instance Segmentation

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Video instance segmentation requires robust object tracking and identity management across frames.
• Existing methods struggle with long-term tracking and dynamic scenes where objects frequently appear and disappear.

Purpose of the Study:

• To introduce a novel system for robustly tracking and continuously updating object states in videos.
• To improve temporally consistent video instance segmentation and long-term instance tracking.
• To enhance object index assignment accuracy and identity consistency in dynamic scenes.

Main Methods:

• Developed Latest Object Memory (LOM) to model object presence across video frames for consistent tracking.
• Introduced Latest Object Memory Management (LOMM) for temporally consistent video instance segmentation.
• Proposed Decoupled Object Association (DOA) to separately handle newly appearing and existing objects for improved matching accuracy.

Main Results:

• The proposed LOMM system achieves state-of-the-art performance in video instance segmentation.
• Achieved a 54.0 AP score on the challenging YouTube-VIS 2022 dataset.
• Demonstrated superior performance over traditional approaches in extensive experiments and ablation studies.

Conclusions:

• The LOM and LOMM systems provide robust and reliable object tracking and identity management in videos.
• The DOA strategy effectively handles dynamic scenes, improving matching accuracy and identity consistency.
• The method sets a new state-of-the-art for video instance segmentation, particularly for long videos.