Effective Techniques for Multimodal Data Fusion: A Comparative Analysis.

Area of Science:

• Robotics
• Machine Learning
• Data Science

Background:

• Robotic data processing faces challenges in creating unified multimodal representations.
• Multimodal learning and data fusion are key to managing vast raw data volumes.

Purpose of the Study:

• To compare three common data fusion techniques: late fusion, early fusion, and sketch.
• To analyze their effectiveness in classification tasks across different data modalities.
• To establish criteria for selecting the optimal fusion technique for specific applications.

Main Methods:

• Experimental comparison of late fusion, early fusion, and sketch techniques.
• Application of fusion methods to classification tasks using diverse datasets (Amazon Reviews, MovieLens).
• Evaluation of model performance based on modality combinations.

Main Results:

• The choice of data fusion technique significantly impacts model performance.
• Optimal modality combination is essential for achieving high performance.
• Effectiveness varies depending on the specific fusion method and data types.

Conclusions:

• Data fusion technique selection is critical for maximizing multimodal model performance.
• Developed criteria to guide the selection of optimal data fusion strategies.
• Highlights the importance of proper modality combination in data fusion for robotics.