A Q-Learning-Based Hyper-Heuristic Genetic Algorithm for Optimizing Human-Robot Collaborative Assembly Lines

Human-robot collaborative assembly line balancing and scheduling constitutes an NP-hard combinatorial optimization problem involving the simultaneous optimization of task assignment, resource allocation, processing mode selection, station-level scheduling, and ergonomic constraints. This study proposes a Q-learning-based hyper-heuristic genetic algorithm (QLHH-GA) to solve the cost-oriented ergonomic mixed-model human-robot collaborative assembly line balancing and scheduling problem. The proposed approach integrates bio-inspired evolutionary mechanisms of population variation and selection with adaptive, Q-learning-guided low-level heuristic selection. The Q-learning layer uses performance feedback to adapt the search strategy to different solution states while maintaining solution feasibility. A mixed-integer linear programming (MILP) model is also developed to minimize the total operating cost, including station opening, labor, robot operation, and energy consumption costs, while enforcing station-level energy expenditure (EE) limits. Computational experiments conducted using benchmark instances of varying sizes and a literature-based industrial case study demonstrate that QLHH-GA produces solutions comparable to those obtained by the MILP model on small-scale instances and maintains strong solution quality on larger instances, for which exact optimization becomes computationally prohibitive. These findings demonstrate the scalability and effectiveness of reinforcement-learning-guided hyper-heuristic search for designing cost-efficient and ergonomically constrained human-robot collaborative assembly lines.