Massive Coordination of Distributed Energy Resources in VPP: A Mean Field RL-Based Bi-Level Optimization Approach

Area of Science:

• Power Systems Engineering
• Artificial Intelligence
• Optimization Theory

Background:

• Coordinating distributed energy resources (DERs) in virtual power plants (VPPs) offers economic and stability benefits.
• Massive coordination of diverse and uncertain DERs presents a significant research challenge.
• Current methods struggle with the scale and heterogeneity of DERs in VPPs.

Purpose of the Study:

• To propose a novel bi-level optimization framework for the massive coordination of DERs in VPPs.
• To address the challenges of heterogeneity and uncertainty in DERs.
• To enhance the efficiency and economic viability of VPP operations.

Main Methods:

• A bi-level optimization approach combining mean-field reinforcement learning (MFRL) and mixed-integer linear programming (MILP).
• Upper-level optimization utilizes MFRL with fast Shapley credit allocation for large-scale coordination.
• Lower-level optimization employs MILP to manage heterogeneous integrated energy systems (IESs).

Main Results:

• The proposed approach significantly improves convergence speed for VPP operation.
• Demonstrates substantial reductions in global operational costs, particularly in massive-scale scenarios.
• Achieved objective improvements of 4.8%-26.6% compared to baseline methods across 10-500 agents.

Conclusions:

• The bi-level optimization approach effectively enables massive coordination of DERs in VPPs.
• MFRL combined with MILP offers a scalable and efficient solution for complex VPP management.
• The method provides significant economic benefits and operational improvements for modern power systems.