ChatTracker: Enhancing Visual Tracking via LLM-Driven Iterative Description Refinement.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Natural Language Processing

Background:

• Vision-Language (VL) trackers leverage natural language for enhanced object tracking.
• Current VL trackers underperform visual trackers due to inaccurate and ambiguous manual annotations.
• Over 10% of annotations in existing VL tracking datasets are inaccurate.

Purpose of the Study:

• To address the annotation quality issue in VL tracking datasets.
• To propose ChatTracker, a novel framework utilizing Multimodal Large Language Models (MLLMs) for high-quality description generation.
• To enhance the accuracy and versatility of VL tracking.

Main Methods:

• Leveraging MLLMs for generating accurate language descriptions.
• Introducing a Reflection-based Language Description Refinement Module for iterative refinement.
• Developing a plug-and-play framework to integrate MLLM-generated descriptions into existing trackers.

Main Results:

• ChatTracker achieves performance comparable to State-of-the-Art (SoTA) trackers.
• Generated descriptions improve VL tracker performance and text-to-image alignment.
• The framework enhances performance in Referring Expression Comprehension (REC), Segmentation (RES), and Referring Video Object Segmentation (R-VOS).

Conclusions:

• ChatTracker effectively addresses annotation inaccuracies in VL tracking.
• The proposed framework demonstrates universality across various visual tasks.
• ChatTracker offers a promising direction for advancing VL tracking and related visual AI applications.