Knowledge Amalgamation for Object Detection With Transformers.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Knowledge amalgamation (KA) is a deep learning technique for transferring knowledge from multiple teacher models to a single student model.
• Existing KA methods are primarily designed for Convolutional Neural Networks (CNNs) and perform poorly when applied to Transformer architectures.
• Transformers are increasingly prevalent in computer vision, necessitating new KA strategies tailored to their unique architecture.

Purpose of the Study:

• To develop an effective knowledge amalgamation scheme specifically for Transformer-based object detection models.
• To address the performance degradation observed when applying traditional KA methods to Transformers.
• To enhance the knowledge transfer process for compact, multi-talented student models utilizing Transformer architectures.

Main Methods:

• Proposed a novel KA scheme for Transformers, dividing it into sequence-level amalgamation (SA) and task-level amalgamation (TA).
• Introduced a sequence-level amalgamation technique that generates hints by concatenating teacher sequences, avoiding redundant aggregation.
• Implemented task-level amalgamation enabling efficient learning of heterogeneous detection tasks via soft targets.

Main Results:

• The proposed sequence-level amalgamation significantly improved student model performance on object detection tasks.
• Traditional KA methods were found to impair student performance when applied to Transformers.
• Transformer-based student models demonstrated rapid mastery of heterogeneous tasks and achieved performance comparable or superior to specialized teacher models.

Conclusions:

• The novel KA approach effectively transfers knowledge to Transformer-based object detection models, overcoming limitations of prior methods.
• Sequence-level amalgamation is crucial for successful knowledge transfer in Transformer architectures.
• Transformer models show strong potential for learning amalgamated knowledge and achieving high performance across diverse tasks.