Conjugate mixture models for clustering multimodal data.

Area of Science:

• Machine Learning
• Computer Vision
• Signal Processing

Background:

• Multimodal clustering involves data from diverse sensors, often lacking direct alignment.
• Independent unimodal clustering struggles with ensuring mutual consistency between modalities.

Purpose of the Study:

• To develop a novel framework for multimodal clustering using conjugate mixture models.
• To address the challenge of ensuring consistency in clustering across different data modalities.

Main Methods:

• Utilized conjugate mixture models that exploit transformations between object and sensor spaces.
• Formulated the problem as a likelihood maximization task.
• Derived and investigated the conjugate expectation-maximization (EM) algorithm, including optimization and initialization strategies.

Main Results:

• Developed a consistent model selection criterion.
• Demonstrated the algorithm's effectiveness in 3D speaker localization using auditory and visual data.
• Investigated convergence properties and proposed optimization techniques to enhance speed.

Conclusions:

• Conjugate mixture models provide a robust framework for multimodal clustering.
• The proposed conjugate EM algorithm offers a consistent and effective solution for integrating data from multiple sensors.
• The method shows promise for real-world applications like multi-sensory localization.