Health state prediction with reinforcement learning for predictive maintenance.

Area of Science:

• Manufacturing Technology
• Artificial Intelligence
• Machine Learning

Background:

• Predictive maintenance is crucial for Industry 4.0, but traditional methods struggle with labeled data and adaptability.
• Reinforcement Learning (RL) learns optimal policies through interaction, bypassing labeled data needs and handling equipment degradation dynamics.

Purpose of the Study:

• To evaluate model-free RL algorithms for predictive maintenance in manufacturing.
• To compare the performance of Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC) in diverse environments.

Main Methods:

• Formulated CNC machine tool wear prediction as a Markov Decision Process (MDP).
• Implemented and tested four model-free RL algorithms: PPO, A2C, DDPG, and SAC.
• Validated performance across four custom environments, analyzing learning dynamics, convergence, and generalization.

Main Results:

• PPO and SAC exhibited the most stable and efficient performance.
• SAC excelled in structured environments, while PPO demonstrated robust generalization capabilities.
• A2C showed consistent long-term learning; DDPG underperformed due to limited exploration.

Conclusions:

• RL shows significant potential for advanced predictive maintenance applications.
• Algorithm selection should align with specific environment characteristics and reward structures for optimal results.