Related Experiment Videos
Deep Reinforcement Learning for Anti-Jamming Dynamic Spectrum Access: A Bootstrap Ensemble Approach with Echo State
Hao Jiang1,2, Xin Bian1, Mingqi Li1
1Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China.
Sensors (Basel, Switzerland)
|August 13, 2026
Summary
This study introduces ESN-BEDQN, a novel dynamic spectrum access (DSA) scheme using deep reinforcement learning. It rapidly adapts to changing jamming patterns, ensuring reliable communication in complex electromagnetic environments.
Area of Science:
- Wireless communication
- Cognitive radio
- Machine learning for spectrum management
Background:
- Dynamic Spectrum Access (DSA) is crucial for efficient spectrum utilization in complex electromagnetic environments.
- Deep Reinforcement Learning (DRL) enhances DSA performance but struggles with generalizing to diverse jamming scenarios.
- External jamming, like swept and comb jamming, poses significant challenges in anti-jamming communication.
Purpose of the Study:
- To propose a novel DSA scheme, ESN-BEDQN, that rapidly adapts to abrupt changes in jamming patterns.
- To integrate an Echo State Network (ESN) for temporal memory and a Bootstrap Ensemble Deep Q-Network (BEDQN) for efficient exploration.
- To develop a jamming pattern change prediction method (Pred-Reset) for timely network resets.
Main Methods:
- Developed ESN-BEDQN by combining ESN's low-complexity temporal memory with BEDQN's diverse readout heads.
- Implemented Pred-Reset using channel idle ratio detection and symmetric KL divergence to predict jamming pattern shifts.
- Utilized simulation to evaluate the performance against conventional Deep Q-Network (DQN) and Long Short-Term Memory (LSTM)-DQN schemes.
Main Results:
- The ESN-BEDQN scheme achieved a near-zero collision rate under periodic jamming.
- ESN-BEDQN demonstrated significantly faster recovery from abrupt jamming pattern changes compared to DQN and LSTM-DQN.
- The Pred-Reset method accurately detected jamming changes and triggered resets, leading to faster convergence.
Conclusions:
- The proposed ESN-BEDQN scheme offers fast and reliable dynamic spectrum access, particularly in environments with sudden jamming pattern variations.
- ESN-BEDQN overcomes the generalization limitations of traditional DRL-based DSA methods in dynamic jamming scenarios.
- The integration of ESN, BEDQN, and Pred-Reset provides a robust solution for anti-jamming communication systems.