The large-scale integration of renewable energy increases operational uncertainty, posing major challenges for voltage and reactive power control. This paper proposes a bi-level multi-agent deep reinforcement learning (MADRL) framework for dual-time-scale optimization in regional power grids. According to the principles of hierarchical and partitioned balance and local compensation of reactive power, upper-level substation agents are modeled within a Markov game to coordinate discrete actions, while a lower-level optimization solver determines continuous variables. To address potential instability caused by infeasible optimization during discrete exploration, a hierarchical reward scheme is introduced to enhance training robustness. By integrating optimization results with reward signals, the framework achieves effective coordination of discrete-continuous and slow-fast control decisions. Simulation results demonstrate that the proposed control strategy significantly outperforms benchmark methods in both decision-making efficiency and solution quality.

voltage and reactive power control; multi-agent; deep reinforcement learning; bi-level optimization framework