Using Large Language Models to Enhance the Reusability of Sensor Data | JoVE Visualize

Area of Science:

Computer Science
Data Science
Internet of Things

Background:

The Internet of Things (IoT) generates massive data volumes, but its potential is limited by data accessibility and interoperability issues.
Sensor data is often locked in proprietary formats or difficult-to-process web formats like HTML, hindering third-party access and reuse.
Regulations like the European Data Act aim to improve data access but do not fully address the technical challenges of data structuring and interoperability.

Purpose of the Study:

To develop and evaluate a methodology for converting raw, non-interoperable sensor data into structured, reusable formats.
To leverage large language models (LLMs) for automated data transformation and structuring.
To demonstrate the effectiveness of this approach using real-world sensor data, specifically meteorological data.

Main Methods:

A novel methodology was developed to extract sensor data from web portals.
Large language models, including GPT-4, were employed to convert sensor data from non-interoperable formats (e.g., HTML) into structured formats (e.g., JSON, XML).
Quantitative and qualitative evaluations were performed using meteorological data to assess the performance of the LLM-based conversion.

Main Results:

The proposed methodology successfully converted raw sensor data into structured formats, enhancing reusability.
GPT-4, a leading LLM, achieved high performance in converting HTML to JSON/XML, with a precision of 93.51% and a recall of 85.33%.
The results confirm the feasibility and effectiveness of using LLMs for overcoming sensor data interoperability challenges.

Conclusions:

Large language models offer a powerful solution for transforming and structuring sensor data, unlocking its potential for data-driven innovation.
The developed methodology significantly improves sensor data reusability by addressing format and interoperability barriers.
This approach paves the way for more accessible and valuable utilization of IoT-generated data across various applications.