A Turbo Q-Learning (TQL) for Energy Efficiency Optimization in Heterogeneous Networks

Area of Science:

• Computer Science
• Electrical Engineering
• Telecommunications

Background:

• Heterogeneous networks (HetNets) face challenges in energy efficiency optimization due to large action and state spaces.
• Existing methods struggle with the complexity of resource configuration in HetNets.

Purpose of the Study:

• To propose a Turbo Q-Learning (TQL) algorithm for maximizing energy efficiency in HetNets.
• To address the large dimensions of action and state spaces in resource configuration optimization.

Main Methods:

• Designed the energy efficiency optimization as a multistage decision process, dividing the problem into subproblems solved by tabular Q-Learning.
• Utilized a deep neural network (DNN) to classify states, overcoming the large state space dimensions.
• Decomposed the exponentially increasing action space into a linear increase for efficient problem-solving.

Main Results:

• The proposed TQL algorithm demonstrates convergence.
• Achieved a 60% improvement in convergence speed compared to traditional methods.
• Maintained comparable energy efficiency with enhanced system adjustability.

Conclusions:

• The TQL approach effectively optimizes resource configuration in HetNets.
• The method successfully handles large state and action spaces, improving efficiency and speed.
• The algorithm offers a stable and adjustable solution for energy efficiency in complex network environments.