Exploring refined dual visual features cross-combination for image captioning.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Transformer-based encoders are standard for image captioning, utilizing multi-head self-attention to capture image context.
• However, standard Transformers incur significant computational overhead due to quadratic complexity in self-attention, leading to redundant feature computation.

Purpose of the Study:

• To propose a novel Distilled Cross-Combination Transformer (DCCT) network for efficient and effective image captioning.
• To address the computational inefficiency and redundant feature issues in existing Transformer-based image captioning models.

Main Methods:

• Introduced a distillation cascade fusion encoder (DCFE) employing probabilistic sparse self-attention to filter redundant features and refine visual representations.
• Developed a parallel cross-fusion attention module (PCFA) to effectively fuse complementary grid and region features.

Main Results:

• The proposed DCCT network demonstrated outstanding performance on the MSCOCO dataset.
• Achieved results competitive with current state-of-the-art image captioning approaches.

Conclusions:

• The DCCT network offers an efficient and effective solution for image captioning by optimizing feature encoding and fusion.
• The novel DCFE and PCFA modules contribute to enhanced performance and reduced computational complexity in Transformer-based models.