Device Data Ingestion for Industrial Big Data Platforms with a Case Study

Area of Science:

• Computer Science
• Data Engineering
• Industrial IoT

Background:

• The Industry 4.0 era relies heavily on the Internet of Things (IoT).
• A key challenge is ingesting large-scale, heterogeneous, and multi-type device data.
• Existing platforms struggle with diverse data streams from industrial environments.

Purpose of the Study:

• To propose a novel heterogeneous device data ingestion model for industrial big data platforms.
• To address the challenge of efficiently processing diverse IoT data.
• To enhance the capabilities of industrial big data platforms.

Main Methods:

• Development of a heterogeneous device data ingestion model.
• Incorporation of device templates for data standardization.
• Implementation of four key strategies: data synchronization, data slicing, data splitting, and data indexing.
• Verification of the model on an industrial big data platform.

Main Results:

• Successful ingestion of device data from multiple sources using the proposed model.
• Demonstrated efficiency in handling heterogeneous and multi-type device data.
• Validation of the model's effectiveness on a real-world industrial big data platform.

Conclusions:

• The heterogeneous device data ingestion model effectively addresses the challenge of large-scale IoT data processing in Industry 4.0.
• The model enhances industrial big data platforms by enabling efficient data ingestion from diverse sources.
• The study provides a practical solution for industrial big data scenario analysis using device data.