Toward Comprehensive Information-Theoretic Multi-View Learning

Area of Science:

• Machine Learning
• Information Theory
• Computer Science

Background:

• Multi-view learning methods often assume redundancy, focusing only on common information between views.
• This assumption overlooks the predictive value of unique information present in individual views.

Purpose of the Study:

• To propose a novel information-theoretic multi-view learning framework (CIML) that leverages both common and unique information.
• To develop a method that discards the restrictive multi-view redundancy assumption.

Main Methods:

• CIML maximizes Gacs-Korner common information for shared features and uses Information Bottleneck (IB) for task-relevant compression.
• IB is applied to unique representation learning, compressing unique information while minimizing its correlation with common and other unique representations.

Main Results:

• Theoretically proved the predictive sufficiency of the learned joint representation.
• Extensive experiments demonstrated CIML's superiority over state-of-the-art multi-view learning methods.

Conclusions:

• CIML offers a more comprehensive approach to multi-view learning by considering both common and unique information.
• The framework advances information-theoretic multi-view learning by moving beyond the redundancy assumption.