A Q-Learning-Enhanced Cuckoo Catfish Optimizer (CCO-RL): A Comparative Study of Nine Metaheuristics Applied to...

Area of Science:

• Artificial Intelligence
• Computational Intelligence
• Optimization Algorithms

Background:

• The Cuckoo Catfish Optimizer (CCO) is a swarm intelligence algorithm with fixed move selection, hindering performance on complex problems.
• Its rigid strategy ignores agent performance and lacks memory, leading to premature convergence and reduced diversity.

Purpose of the Study:

• To improve the Cuckoo Catfish Optimizer's adaptability and performance on high-dimensional problems.
• To introduce a reinforcement learning agent for dynamic strategy selection within the CCO framework.

Main Methods:

• Developed CCO-RL by integrating a tabular Q-learning controller to dynamically select CCO movement strategies.
• The controller uses a 48-state summary of agent behavior (crowding, stagnation, progress) to inform move selection.
• Employed a bounded reward and decaying epsilon-greedy policy for online learning without additional function evaluations.

Main Results:

• CCO-RL achieved the best overall Friedman rank (1.69) across 70 benchmark instances from CEC2017 and CEC2022.
• Statistical analysis (Nemenyi test) confirmed CCO-RL significantly outperformed eight popular metaheuristics.
• The enhanced algorithm also secured the best mean design for three engineering problems.

Conclusions:

• CCO-RL effectively addresses the limitations of fixed strategy selection in swarm intelligence algorithms.
• Reinforcement learning integration provides a robust and adaptive mechanism for optimizing complex, high-dimensional search spaces.
• The proposed method demonstrates superior performance and adaptability compared to existing optimization techniques.