Action-factorized Rainbow deep Q-network with token Transformer for computation offloading in edge computing-enabled...

Area of Science:

• Maritime technology
• Computer science
• Artificial intelligence

Background:

• Edge computing (EC) in the Internet of Ships (IoS) offers reduced latency and energy consumption compared to cloud architectures.
• Effective computation offloading is crucial for realizing EC benefits but is challenging in dynamic maritime environments due to complex decision spaces and variable wireless channels.

Purpose of the Study:

• To propose a deep reinforcement learning (DRL) algorithm for discovering efficient computation offloading strategies in EC-enabled IoS (EC-IoS).
• To address the challenges of high-dimensional decision spaces, system constraints, and dynamic maritime wireless channels.

Main Methods:

• Development of an action-factorized Rainbow deep Q-network (DQN) incorporating a token Transformer.
• Custom token Transformer-based state and action encoders to manage complex decision spaces.
• Acceleration using a parallel training architecture for improved learning efficiency and stability.

Main Results:

• The proposed algorithm's learned computation offloading strategies significantly outperform baseline methods on the weighted latency-energy objective.
• Achieved a zero rate of invalid actions, ensuring all system constraints are met and practical feasibility.
• Demonstrated robust performance in balancing latency and energy consumption.

Conclusions:

• The action-factorized Rainbow DQN with token Transformer provides a robust solution for computation offloading in EC-IoS.
• The algorithm effectively supports maritime digitalization and automation by optimizing performance and ensuring practical implementation.
• Highlights the potential of DRL in solving complex optimization problems in maritime edge computing environments.