Fashion-Oriented Image Captioning with External Knowledge Retrieval and Fully Attentive Gates.

Area of Science:

• Computer Vision
• Multimedia Processing
• Natural Language Processing

Background:

• Fashion and e-commerce research is growing in computer vision and multimedia.
• Generating fine-grained, accurate natural language descriptions for fashion items is an under-explored challenge.

Purpose of the Study:

• To develop an advanced model for fine-grained fashion image captioning.
• To overcome limitations of existing approaches in generating accurate fashion descriptions.

Main Methods:

• A transformer-based captioning model integrated with external textual memory.
• Utilized k-nearest neighbor (kNN) searches for memory retrieval.
• Implemented cross-attention operations and a novel fully attentive gate for information flow control.

Main Results:

• The proposed model demonstrated superior performance on the fashion captioning dataset (FACAD).
• Experimental validation confirmed the effectiveness of the architectural strategies.
• The method consistently outperformed baseline and state-of-the-art approaches.

Conclusions:

• The developed transformer model with external memory is highly effective for fashion image captioning.
• The novel architectural components significantly enhance description accuracy.
• This research advances the state-of-the-art in fine-grained fashion item description generation.