Real-time black-box optimization for dynamic discrete environments using embedded Ising machines

Many real-time systems require the optimization of discrete variables. Black-box optimization (BBO) algorithms and multi-armed bandit (MAB) algorithms perform optimization by repeatedly taking actions and observing the corresponding immediate rewards without any prior knowledge. Recently, a BBO method using an Ising machine has been proposed to find the best action represented by a combination of discrete values and maximize the immediate reward in static environments. By contrast, real-time systems operate in dynamic environments and necessitate MAB algorithms that maximize the average reward over repeated trials. Due to the enormous number of actions resulting from the combinatorial nature of discrete optimization, conventional MAB algorithms cannot effectively optimize actions for dynamic, discrete environments. Here, we show a heuristic method to maximize the average reward for dynamic discrete environments by extending the BBO method, in which an Ising machine efficiently explores actions while accounting for interactions between variables and environmental changes. We demonstrate the adaptability to dynamic environments of the proposed method in a wireless communication system with moving users.