Pro2Diff: Proposal Propagation for Multi-Object Tracking via the Diffusion Model.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-object tracking (MOT) involves estimating object bounding boxes and ID labels in videos, facing challenges in balancing detection and tracking subtasks.
• Existing methods include two-stage Tracking-By-Detection (TBD) and single-stage Joint Detection and Tracking (JDT), each with limitations in architecture complexity or optimization.
• Competitive learning between detection and tracking subtasks remains a key challenge in MOT.

Purpose of the Study:

• To propose a new MOT method, Pro2Diff (Proposal Propagation via Diffusion Models), integrating diffusion models into the proposal propagation process.
• To enhance MOT performance by focusing on the model training process, avoiding complex network design modifications.
• To explore the effectiveness of generative approaches and self-conditional proposal propagation in MOT.

Main Methods:

• Pro2Diff utilizes a generative approach with diffusion models, creating noisy proposals in a forward process for video sequences.
• The method learns discrepancies between noisy proposals and actual object bounding boxes, optimizing proposals through a denoising diffusion process.
• Self-conditional proposal propagation is introduced to improve model performance during inference without altering the model structure.

Main Results:

• Pro2Diff demonstrates that generative methods can effectively address multi-object tracking tasks.
• Self-conditional proposal propagation enhances model performance during inference without structural modifications.
• Optimal performance is achieved by adjusting proposal numbers and iterations based on tracking sequences.
• Experimental results on MOT17 and DanceTrack datasets show Pro2Diff outperforms current end-to-end MOT methods, achieving 61.9 HOTA on DanceTrack and 57.6 HOTA on MOT17.

Conclusions:

• Generative diffusion models offer a promising direction for advancing multi-object tracking.
• Pro2Diff provides a competitive and effective alternative to existing JDT approaches by optimizing the training process.
• The proposed method achieves state-of-the-art results, highlighting the potential of diffusion models in MOT.