OffsetNet: Towards Efficient Multiple Object Tracking, Detection, and Segmentation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Offset-based representations are effective for pixel and motion modeling in computer vision.
• Existing methods often address object detection, segmentation, and tracking separately.

Purpose of the Study:

• To introduce OffsetNet, a novel one-stage multi-tasking network for Multi-Object Tracking and Segmentation (MOTS).
• To extend the offset-based representation to concurrently handle amodal bounding box detection, instance segmentation, and tracking.

Main Methods:

• Developed a unified pixel-offset-based representation for concurrent task execution.
• Incorporated a Memory Enhanced Linear Self-Attention (MELSA) block for efficient spatial-temporal feature aggregation.
• Utilized three lightweight decoders for one-shot task decoupling and a cross-frame offsets prediction module for occlusion robustness.

Main Results:

• Achieved 76.83% HOTA on the KITTI MOTS benchmark without 3D detection.
• Reached 74.83% HOTA at 50 FPS on the KITTI MOT benchmark, outperforming CenterTrack.
• Demonstrated significant speed improvements (3.3x faster than CenterTrack) with enhanced performance.

Conclusions:

• OffsetNet offers an efficient and effective unified framework for MOTS tasks.
• The proposed MELSA block and cross-frame offset prediction enhance feature aggregation and tracking robustness.
• OffsetNet establishes a strong baseline for future research in multi-object tracking and segmentation.