A Deep Multi-Modal CNN for Multi-Instance Multi-Label Image Classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep convolutional neural networks (CNNs) excel at single-label image classification but struggle with multi-label tasks.
• Existing methods often treat images as single instances, failing to capture diverse visual information and label correlations.

Purpose of the Study:

• To propose a novel deep multi-modal CNN for multi-instance multi-label image classification (MMCNN-MIML).
• To address limitations in handling mixed visual information and label correlations in multi-label image classification.

Main Methods:

• The MMCNN-MIML model combines CNNs with multi-instance multi-label (MIML) learning, representing images as bags of instances.
• It automatically generates instance representations, groups labels to leverage correlations, and incorporates textual context for multi-modal instances.

Main Results:

• MMCNN-MIML demonstrates superior performance on benchmark multi-label image datasets.
• The model effectively discriminates visually similar objects belonging to different label groups.

Conclusions:

• The proposed MMCNN-MIML significantly advances multi-label image classification by integrating CNNs and MIML learning.
• This approach offers a robust solution for complex image classification tasks with multiple labels and instances.