Efficient Image and Sentence Matching.

Area of Science:

• Computer Vision
• Natural Language Processing
• Machine Learning

Background:

• Large models for image-sentence matching achieve high accuracy but are computationally expensive.
• These large models require significant storage and have slow inference speeds, limiting their use on low-cost devices.
• There is a need for efficient yet accurate models for real-world image and sentence matching applications.

Purpose of the Study:

• To develop an efficient method for image and sentence matching.
• To distill knowledge from a large teacher model to a smaller, more efficient student model.
• To maintain high accuracy in the distilled student model.

Main Methods:

• Proposed Whitened Similarity Distillation (WSD) for cross-modal knowledge distillation.
• Employed efficient backbone networks for feature representation.
• Utilized fast N-to-N similarity measurement.
• Applied whitening-like transformations to reduce variation inconsistency between cross-modal similarity matrices.

Main Results:

• The distilled student model is 7x smaller and 9x faster than the teacher model.
• Achieved only a 2% decrease in accuracy compared to the teacher model.
• Demonstrated the effectiveness of WSD on two benchmark datasets.

Conclusions:

• WSD is an effective method for creating efficient and accurate image-sentence matching models.
• The proposed approach successfully distills knowledge from large to small models.
• WSD offers a practical solution for deploying image-sentence matching on resource-constrained devices.