Fog-assisted de-duplicated data exchange in distributed edge computing networks

Area of Science:

• Computer Science
• Data Science
• Network Engineering

Background:

• Internet of Things (IoT) systems generate vast amounts of sensor data, posing challenges for resource-scarce devices.
• Inefficient data handling in IoT leads to increased communication and storage costs due to redundant transmissions from overlapping sensor areas.
• Existing data deduplication schemes like Asymmetric Extremum (AE) and Rapid Asymmetric Maximum (RAM) struggle with optimal window size selection, leading to poor deduplication rates.

Purpose of the Study:

• To address the limitations of existing data deduplication techniques in resource-constrained IoT environments.
• To propose a novel algorithm, the Controlled Cut-point Identification Algorithm (CCIA), for efficient data chunking and deduplication.
• To enhance data collection, transmission, and storage efficiency in IoT-enabled healthcare services and other applications.

Main Methods:

• Developed the Controlled Cut-point Identification Algorithm (CCIA) with a restricted variable-sized window to a defined threshold.
• Ensured the index value for threshold determination is greater than half the fixed window size to maximize duplicate detection.
• Implemented an upper limit offset to prevent excessively large windows and mitigate computational overhead.
• Conducted extensive simulations using Windows Communication Foundation services deployed on the Azure cloud platform.

Main Results:

• CCIA demonstrated superior performance compared to AE and RAM across multiple metrics, including chunk number, average chunk size, and minimum chunk size.
• CCIA achieved significant improvements in total number of chunks (6.81% and 14.17% better than competitors) and average number of chunks (4.39% and 18.45% better).
• The algorithm showed substantial gains in minimum chunk size (153% and 190% improvement), indicating more effective data compression and deduplication.

Conclusions:

• The Controlled Cut-point Identification Algorithm (CCIA) effectively optimizes data transmission and storage in IoT systems.
• CCIA offers improved resource utilization and reduced operational costs for IoT applications, particularly in data-intensive fields like healthcare.
• The proposed algorithm represents a significant advancement in efficient data management for resource-constrained IoT environments.