Towards Deep Q-Network Based Resource Allocation in Industrial Internet of Things

With the increasing adoption of Industrial Internet of Things (IIoT) devices, infrastructures, and supporting applications, it is critical to design schemes to effectively allocate resources (e.g., networking, computing, and energy) in IIoT systems, generally formalized as optimization problems. Nonetheless, because the system is highly complex, operation environments are time-varying, and required information may not be available, it is difficult to leverage traditional optimization techniques to solve the optimal resource allocation problem. To this end, in this paper we propose a Deep $Q$-Network (DQN) based scheme to address both bandwidth utilization and energy efficiency in an IIoT system. In detail, we design a DQN model that consists of two deep neural networks (DNN) and a $Q$-learning model. The DNN network abstracts the features from the highly dimensional inputs and obtains the approximate $Q$-function for the $Q$-learning model. Based on the $Q$-function, the $Q$-learning model can generate the $Q$-table and reward function. After the training process, the DQN model can select appropriate actions for the agents (i.e., robots in a smart warehouse in this study) to improve bandwidth utilization and energy efficiency. To evaluate our proposed scheme, we design a simulation environment to investigate a typical IIoT scenario: the actuation of robotics in a smart warehouse. We then implement the DQN model and conduct extensive experiments to validate the efficacy of our scheme. Our experimental results confirm that our scheme can improve both bandwidth utilization and energy efficiency, as compared to other representative schemes.