A Tactile Cognitive Model Based on Correlated Texture Information Entropy and Multimodal Fusion Learning

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Perception

Background:

• Advancing robotic dexterity requires realistic tactile cognition, hindered by limited texture datasets and complex signal fusion.
• Existing methods struggle with the heterogeneity of multimodal tactile signals and dataset realism.

Purpose of the Study:

• To develop a comprehensive framework for multimodal tactile cognition.
• To overcome limitations in texture dataset realism and signal fusion for enhanced robotic perception.

Main Methods:

• A universal texture dataset was created using information entropy and Perlin noise for broad surface simulation.
• The Multimodal Fusion Attention Transformer Network (MFT-Net) was designed, integrating CNNs, Transformers, and attention mechanisms.
• MFT-Net employs Convolutional Neural Networks (CNNs) for local features and Transformers for global dependencies, with Squeeze-and-Excitation attention for cross-modal weighting.

Main Results:

• MFT-Net achieved 86.66% classification accuracy on the custom dataset, exceeding baselines by over 21.99%.
• An information-theoretic analysis demonstrated a correlation between texture information content and model recognition performance.
• The developed dataset proved effective in evaluating and improving tactile model generalization.

Conclusions:

• A novel design-verification paradigm linking physical information to machine perception was established.
• This work provides a quantifiable method to enhance tactile model generalization for complex robotic tasks.
• The framework paves the way for improved robotic dexterity in diverse, real-world environments.