LLM-guided population-based reinforcement learning: A scalable methodology for adaptive hyperparameter optimization

Population-Based Training (PBT) has the drawback of using fixed, pre-programmed mutation and selection rules to optimize hyperparameters, which are not always flexible across reinforcement learning (RL) tasks. To address this, we introduce LLM-Guided Population-Based Reinforcement Learning (LPBRL), a scalable methodology in which the reasoning capability of Large Language Models (LLMs) is used to manage population evolution dynamically. LPBRL operates through a six-phase cycle in which the LLM analyzes real-time performance measurements from parallel workers and produces adaptive population-update recommendations as a substitute for static rules. In contrast to conventional PBT, and unlike prior LLM-assisted optimization frameworks that typically operate outside the recurrent population loop, LPBRL places language-model reasoning directly inside the selection-mutation stage of training. This enables task-aware hyperparameter adaptation that improves convergence speed and training stability. We evaluated LPBRL on CartPole-v1 with 8 parallel workers over 150 episodes and observed clear gains over conventional PBT, with best- and average-reward convergence improving by 62.5 percent and 68.2 percent, respectively. Although the approach requires access to LLM APIs and compatible RL tooling such as Stable-Baselines3, the results show strong potential for large-scale training workflows in which adaptive hyperparameter control is essential. Overall, the empirical findings support the claim that language-model reasoning can make effective optimization decisions in RL while preserving the practical strengths of population-based training.•Large Language Models are integrated as adaptive decision-makers inside the recurrent population-evolution loop, replacing static task-agnostic mutation and selection rules with context-aware reasoning.Real-time worker metrics, trajectory trends, and LLM-guided hyperparameter adaptation accelerate convergence and improve stability across discrete and continuous-control RL settings.•The methodology provides a reproducible implementation path with structured prompts, deterministic parsing, bounded updates, and compatibility with multiple RL algorithms (PPO, SAC, TD3), supporting large-scale applications.