Sensor Fusion for Target Detection Using LLM-Based Transfer Learning Approach.

Area of Science:

• Robotics and Autonomous Systems
• Artificial Intelligence
• Sensor Fusion

Background:

• Traditional sensor fusion methods often rely on theoretical models, treating sensors as independent.
• Real-world data presents challenges like noise and imperfections, impacting autonomous agent performance.
• Integrating data from diverse sensors like optical and LIDAR remains a significant hurdle.

Purpose of the Study:

• To introduce a novel sensor fusion approach for detecting multiple static and mobile targets using autonomous mobile agents.
• To leverage real-world sensor data and integrate it via a large language model (LLM) framework.
• To improve target detection precision, recall, and computational efficiency in challenging environments.

Main Methods:

• Developed a methodology based on LLM transfer learning (LLM-TLFT) to create a global probability map.
• Transformed real-world optical and LIDAR sensor data into sensor-specific probability maps using deep learning.
• Integrated probability maps through an LLM framework, enabling interpretation of sensor dependencies.

Main Results:

• Demonstrated significant performance improvements over existing methods (Independent Opinion Pool, CNN, GPT-2 with deep transfer learning).
• Achieved higher precision and recall, particularly in scenarios with high noise and sensor imperfections.
• Successfully implemented model compression via knowledge-based distillation (distilled TLFT) for edge device deployment.

Conclusions:

• The proposed LLM-TLFT approach offers a robust solution for multi-target detection by autonomous agents.
• The method effectively handles real-world sensor data, including noise and imperfections, outperforming conventional techniques.
• The approach facilitates efficient swarm management and enables deployment on edge devices through model compression.