Effective Pre-Training Method and Its Compositional Intelligence for Image Captioning.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Deep Learning

Background:

• Deep learning models are growing exponentially in parameter size, increasing computation and training costs.
• High training costs hinder the development and deployment of advanced AI models.

Purpose of the Study:

• To introduce Compositional Intelligence (CI), a novel method for reducing deep learning training costs.
• To evaluate the effectiveness of CI in the image captioning task by combining pre-trained models.

Main Methods:

• Proposed Compositional Intelligence (CI), a model reuse strategy combining pre-trained models.
• Applied CI to the image captioning task, utilizing pre-trained feature extractors and caption generators (Transformer model).
• Compared CI models against 'From Scratch' training using early stopping on the MS-COCO dataset.

Main Results:

• The vanilla image captioning model using CI reduced training cost by 13.8% and improved performance by 3.2%.
• The Object Relation Transformer model with CI achieved a 21.3% reduction in training cost.
• CI demonstrated significant reductions in training cost and performance improvements.

Conclusions:

• Compositional Intelligence (CI) offers an effective approach to mitigate the high training costs associated with large deep learning models.
• Reusing pre-trained components via CI leads to substantial savings in computation and time while enhancing model performance in tasks like image captioning.