Learning a Conceptual Space as a Bridge for Text-Video Retrieval

The goal of text-video retrieval is to learn a shared embedding space in which semantically relevant videos and texts have higher similarity. Recent advances employ contrastive learning to bring the paired video and text features closer. Despite these achievements, the inherent modality disparity, i.e., videos are detailed while texts are often brief, leads to the issues of incomplete correspondence and false negatives between video and text. These issues limit the effectiveness of existing contrastive learning approaches that rely on hard labels for direct text-video alignment. To this end, we present CoSTVR, a novel framework that introduces a conceptual space as a semantic bridge to enhance text-video feature alignment. Specifically, we regard concepts as a shared representation that bridges video and text modalities, and construct a conceptual codebook based on a set of concept prototypes. Building upon this codebook, we perform complementary concept-centered alignment in addition to direct text-video alignment. To implement concept-centered alignment from different feature granularities, we design token quantization and concept perception modules. Token quantization maps continuous patch-level and word-level features to discrete conceptual embeddings. Concept perception predicts the concepts present in video and text from global features, supervised by the concept labels we constructed. These two modules guide video and text features toward shared conceptual representations, grounding both modalities in a unified conceptual space. Extensive experiments demonstrate the effectiveness of CoSTVR, achieving state of-the-art performance on three popular benchmarks, MSRVTT, DiDeMo, and ActivityNet Captions. Further ablation analysis shows that our method effectively mitigates the modality gap and enhances text-video alignment.