Related Experiment Videos
Multisource Sensor Fusion and Large Language Model Integration for Explainable State Perception and Anomaly Awareness
Bocheng Zhou1,2, Jinze Xie1, Tiantian Chen2
1China Agricultural University, Beijing 100083, China.
Sensors (Basel, Switzerland)
|August 13, 2026
Summary
This study introduces a new framework for artificial intelligence-driven sensing that analyzes language and behavior consistency. It effectively detects subtle risks by fusing multisource data, improving system monitoring and early warning capabilities.
Area of Science:
- Artificial Intelligence
- Data Science
- Systems Engineering
Background:
- Intelligent sensing systems generate heterogeneous data, necessitating advanced analysis for operational state identification and risk detection.
- Existing methods often focus on text or behavior data separately, missing crucial insights from their joint analysis.
- Timely detection of subtle risks, indicated by deviations between stated intentions and actual behavior, remains a challenge.
Purpose of the Study:
- To propose a novel language-behavior consistency sensing framework for multisource sensing signals.
- To enable intelligent perception and quantitative analysis of deviations between textual declarations and executed behaviors.
- To improve the accuracy and timeliness of complex system state identification and risk perception.
Main Methods:
- Developed a framework to map textual and behavioral sensing signals into a shared state logic space.
- Implemented modules for textual state logic extraction, observed behavioral state modeling, and language-behavior consistency measurement.
- Conducted experiments on a diverse multisource sensing dataset including texts, logs, and resource records.
Main Results:
- The proposed language-behavior consistency sensing framework achieved superior performance compared to baseline methods.
- Key performance metrics included a language-behavior consistency score (LCS) of 0.742, AUC of 0.846, and F1-score of 0.811.
- The framework demonstrated strong explanation consistency score (ECS) of 0.821, outperforming advanced multimodal models.
Conclusions:
- Language-behavior consistency sensing effectively fuses multisource information for improved complex system monitoring.
- The framework offers an interpretable approach for anomaly early warning and risk perception in intelligent sensing.
- Potential applications include industrial operations, intelligent manufacturing, and digital infrastructure management.