Visual-Tactile Fused Graph Learning for Object Clustering.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Existing object clustering methods often rely solely on visual data, neglecting tactile information, which limits performance.
• Simple concatenation of visual and tactile data in multiview clustering fails to fully leverage complementary information due to inherent modality differences.

Purpose of the Study:

• To develop a graph-based framework for object clustering that fuses visual and tactile information effectively.
• To address limitations of existing methods by exploring complementary information between modalities and mitigating their differences.

Main Methods:

• A graph-based visual-tactile fused object clustering framework with two modules: Modality-specific Representation learning (M_R) and Unified Affinity Graph learning (M_U).
• M_R uses deep non-negative matrix factorization (NMF) and an autoencoder-like structure for robust, compact, modality-specific representations.
• M_U employs a minimizing disagreement scheme to align representations and a Laplacian rank constraint for regularization, enabling direct label acquisition.

Main Results:

• The proposed framework demonstrates superior performance in object clustering across five public datasets.
• The integration of visual and tactile data through the proposed method significantly improves clustering accuracy compared to existing approaches.
• The framework effectively mitigates differences between visual and tactile modalities, maximizing mutual information for better clustering outcomes.

Conclusions:

• The developed graph-based framework offers a significant advancement in object clustering by effectively fusing visual and tactile information.
• The method provides a robust and efficient approach to object clustering, outperforming existing techniques.
• This research highlights the potential of multimodal fusion for enhancing machine perception tasks.