Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

Area of Science:

• Cyberphysical Systems (CPS)
• Internet of Things (IoT)
• Data Management

Background:

• Digital twins require timely synchronization of physical attributes with digital spaces.
• Data uncertainty arises from limitations in sampling and observing physical attributes in CPS.
• Integrating real-world objects and digital assets faces challenges in current CPS architectures.

Purpose of the Study:

• To propose a learning-based data management scheme to mitigate data uncertainty in CPS.
• To introduce a sensor data management framework, D2WIN, for enhancing data quality.
• To address the scalability issues of reinforcement learning (RL) in managing numerous sensor streams.

Main Methods:

• Developed the D2WIN framework utilizing reinforcement learning (RL) techniques.
• Proposed an action embedding strategy based on physical space coordination for RL scalability.
• Implemented two embedding methods: a user-defined function and a generative model.

Main Results:

• D2WIN with action embedding outperformed existing heuristics in data quality under resource constraints.
• The framework demonstrated significant benefits in an autonomous driving simulator.
• The driving agent maintained 96.2% performance with only 30% of sensor updates applied.

Conclusions:

• The D2WIN framework effectively manages sensor data quality for CPS applications.
• Action embedding is crucial for scaling RL-based data management in complex systems.
• The proposed approach offers a practical solution for reducing data uncertainty in digital twin implementations.