Video Object Segmentation Using Kernelized Memory Network With Multiple Kernels.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Semi-supervised video object segmentation (VOS) utilizes initial frame masks for target segmentation in subsequent frames.
• Space-time memory networks (STM) are a recent advancement in semi-supervised VOS.
• A key challenge is the inherent non-local nature of STM applied to the predominantly local VOS problem.

Purpose of the Study:

• To propose novel VOS networks that reconcile the non-local nature of STM with the local characteristics of VOS.
• To introduce Kernelized Memory Networks (KMN) and KMN with multiple kernels (KMN^M).
• To improve occlusion handling through a pre-training strategy.

Main Methods:

• Developed KMN and KMN^M incorporating both Query-to-Memory and Memory-to-Query matching.
• Introduced kernel-based Memory-to-Query matching to reduce non-localness in STM.
• Implemented a Hide-and-Seek strategy for pre-training to enhance robustness against occlusions.

Main Results:

• The proposed KMN and KMN^M significantly outperform state-of-the-art methods, achieving a +4% improvement in J_M on the DAVIS 2017 test-dev set.
• Achieved competitive inference speeds of 0.12 and 0.13 seconds per frame for KMN and KMN^M, respectively, comparable to STM.
• Demonstrated effective handling of occlusions through the Hide-and-Seek pre-training strategy.

Conclusions:

• KMN and KMN^M effectively address the non-local/local mismatch in semi-supervised VOS.
• The proposed methods offer superior performance and efficiency compared to existing STM approaches.
• The Hide-and-Seek strategy enhances robustness, making KMN suitable for real-world VOS applications.