Deep Multi-View Learning using Neuron-Wise Correlation-Maximizing Regularizers.

Area of Science:

• Machine Learning
• Computer Vision
• Deep Learning

Background:

• Multi-view learning aims to find common patterns across diverse data modalities.
• Deep multi-view networks often fuse features from individual views at intermediate layers.
• Existing methods adapt generic Deep Neural Networks (DNNs) for multi-view tasks.

Purpose of the Study:

• To develop a novel regularization approach for end-to-end deep multi-view learning.
• To propose an effective and efficient neuron-wise correlation-maximizing regularizer.
• To improve performance on complex multi-view recognition tasks.

Main Methods:

• Introduced a correlation-regularized network layer (CorrReg) as a plug-in module.
• CorrReg maximizes neuron-wise correlations within fusion layers of DNNs.
• Applied CorrReg to fully-connected and convolutional fusion layers.

Main Results:

• CorrReg consistently improved classification performance across various multi-view learning problems.
• Achieved new state-of-the-art results on benchmark RGB-D object and scene recognition datasets.
• Demonstrated efficacy through control experiments on image classification tasks.

Conclusions:

• The proposed CorrReg is an effective regularization technique for deep multi-view learning.
• CorrReg offers a practical method to enhance feature learning and classification accuracy.
• The implementation of CorrReg is publicly available for further research and application.