Comparison of Input-Data Matrix Representations Used for Continual Learning with Orthogonal Weight Modification on...

Area of Science:

• Artificial Intelligence
• Industrial Internet of Things (IIoT)
• Machine Learning

Background:

• Industrial processes increasingly use smart devices for automation and optimization.
• Industrial Internet of Things (IIoT) facilitates device communication but lacks advanced process optimization.
• Object detection sensors are key components in smart industrial applications.

Purpose of the Study:

• To explore Artificial Neural Twin (ANT) as a distributed optimization tool for industrial processes.
• To investigate the integration of continual learning (CL) methods like Orthogonal Weight Modification (OWM) for autonomous device learning.
• To compare matrix approximation methods for reducing computational complexity in CL algorithms on resource-constrained devices.

Main Methods:

• Utilized an object detection sensor as a testbed for ANT and OWM.
• Implemented and compared Fisher matrix, NEig-OWM, and LoRA for matrix approximation in CL.
• Evaluated the trade-offs between computational cost, hardware requirements, and model performance.

Main Results:

• The Fisher matrix proved to be the most computationally inexpensive approximation for CL.
• Negligible performance reduction was observed when using the Fisher matrix for CL in large AI models.
• NEig-OWM demonstrated suitability for smaller models requiring greater control over the CL process.

Conclusions:

• The Fisher matrix is a viable and cost-effective solution for enabling continual learning in large-scale industrial AI systems.
• NEig-OWM offers a more controlled approach to continual learning for resource-limited microcontrollers.
• ANT combined with efficient CL matrix approximations can significantly advance autonomous process optimization in IIoT environments.