Related Experiment Video
Updated: Jul 20, 2026
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
Published on: March 2, 2015
Efficient reinforcement learning for urban drainage control via a neural network-based model using truncated and
Zhiyu Zhang1, Wenchong Tian2, Zhenliang Liao3
1School of Energy and Environment, City University of Hong Kong, Hong Kong Special Administrative Region of China; College of Environmental Science and Engineering, Tongji University, Shanghai, 200092, China; City University of Hong Kong Shenzhen Research Institute, Shenzhen, China; State Key Laboratory of Marine Environmental Health, City University of Hong Kong, Hong Kong Special Administrative Region of China.
Reinforcement learning (RL) for urban drainage networks is now faster and more reliable. A new truncated and parallel rollout framework using neural network models significantly reduces training time while maintaining effective control policies.
Area of Science:
- Environmental Engineering
- Artificial Intelligence
- Water Resource Management
Background:
- Reinforcement learning (RL) shows promise for real-time control of urban drainage networks.
- However, extensive simulations make RL training computationally expensive and prone to errors from surrogate models.
Purpose of the Study:
- To develop an efficient and reliable RL training framework for urban drainage networks.
- To address the computational cost and policy unreliability issues in current RL applications.
Main Methods:
- Proposed a truncated and parallel rollout framework for RL training.
- Utilized a neural network-based model (NNM) for simulations within limited horizons.
- Benchmarked NNM-based RL against physics-based model (PBM)-based RL for control agent development.
Main Results:
- NNM-based RL achieved reliable training and effective control policies comparable to PBM-based RL.
- Reduced agent training time by over 30-fold using a GPU-enabled parallel workflow.
- Carefully selected truncated rollout horizon balanced model errors and hydraulic responses.
Conclusions:
- The truncated and parallel rollout framework enhances the feasibility and cost-effectiveness of RL for real-time control of drainage networks.
- This approach mitigates error accumulation and improves training efficiency without sacrificing control performance.