X-CoT: Explainable Text-to-Video Retrieval via LLM-based Chain-of-Thought Reasoning

Area of Science:

• Computer Science
• Artificial Intelligence
• Information Retrieval

Background:

• Current text-to-video retrieval relies on embedding models and cosine similarity.
• This approach struggles with low-quality data and lacks interpretability in rankings.
• Assessing retrieval models and data quality is challenging with existing methods.

Purpose of the Study:

• To develop an explainable retrieval framework for text-to-video systems.
• To enable assessment of retrieval models and examination of text-video data quality.
• To replace embedding-based similarity ranking with interpretable reasoning.

Main Methods:

• Proposing X-CoT, a framework utilizing Large Language Model Chain-of-Thought (LLM CoT) reasoning.
• Expanding existing benchmarks with enhanced video annotations to mitigate data bias.
• Implementing a retrieval CoT with pairwise comparison for detailed reasoning and ranking.

Main Results:

• X-CoT demonstrates empirical improvements in text-to-video retrieval performance.
• The framework generates detailed rationales for ranking outcomes.
• Facilitates analysis of model behavior and the quality of text-video data pairs.

Conclusions:

• X-CoT offers an interpretable alternative to traditional embedding-based retrieval.
• The framework enhances retrieval accuracy while providing insights into model and data.
• Enables more robust evaluation and understanding of text-to-video retrieval systems.