Automated deep learning by recurrent hyperparameter optimization

Optimizing hyperparameters of deep learning models for specific tasks requires substantial domain expertise and computational resources, remaining challenging in automated deep learning. Existing hyperparameter optimization (HPO) methods are restricted to limited discrete hyperparameter types, rely on manual priors, and fail to scale to large datasets. This paper presents Rocket, a recurrent HPO framework that automates the tuning of mixed-type hyperparameters by self-play reinforcement learning, requiring no prior domain knowledge. A policy agent is developed to learn from historical experience and progressively refine its strategy through iterative interactions with the target model. To address severe reward delay on large-scale datasets, a reward approximation mechanism is designed for data subsets, accelerating policy learning by up to 80X. Across 8 deep learning tasks and 32 benchmarks, Rocket enables target models to achieve state-of-the-art performance from scratch, matching expert-tuned results. In real industrial deployment, Rocket reduces optimization time by 13.4-fold and cost by 73%.