Discovering Visual Co-occurrence for Fine-grained Image Retrieval

Recent fine-grained image retrieval work predominantly focuses on learning diverse visual patterns from training set and further improving the generalization capability when confronted with unseen categories. However, such a design under-exploits the compositional structure of fine-grained objects, leading to rare semantics being easily overwhelmed by more universal ones, such as large-range bird feathers, and even overlooked. Consequently, it fails to identify unknown subcategories characterized by these rare semantics, thereby decreasing retrieval performance. As a remedy, we propose a novel Visual Co-occurrence Discovery Network (VCDN) to discover semantic co-occurring relationships within objects and facilitate the inference of their compositional structures. VCDN leverages the object's compositional structures to ensure equal representation of diverse semantics, reducing the dominance of more universal semantics and highlighting rare ones. Technically, VCDN involves initially employing a sliding-window sampling strategy to uniformly extract local regions from images. From these regions, visual semantics are distilled to identify comprehensive visual representation within objects. Recognizing the disparity in saliency within these semantics, VCDN establishes local intra-semantic correspondences to adjust their saliency levels, leveraging complementary information from semantically related concepts. Furthermore, co-occurring relationships among diverse semantics are dynamically modeled through grouping and interaction, enabling precise inference of the object's compositional structure and emphasizing rare semantics. Quantitative and qualitative experiments show that VCDN achieves promising performance on fine-grained and product retrieval datasets.