Optimizing dense feed-forward neural networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models offer powerful learning capabilities but designing efficient network architectures remains a significant challenge.
• Complex models often lead to high computational costs, large memory footprints, and environmental concerns, limiting their use on resource-constrained devices.

Purpose of the Study:

• To propose a novel method for constructing dense feed-forward neural networks that are both efficient and effective.
• To address the challenges of computationally intensive models and large memory footprints in deep learning.

Main Methods:

• A new method for constructing dense feed-forward neural networks based on pruning and transfer learning.
• Performance evaluation through classification and regression tasks, assessing parameter compression and accuracy retention.

Main Results:

• The proposed method achieves over 70% compression in the number of parameters without any loss in accuracy.
• Refined models, with careful parameter selection, often outperform the original complex models.
• The method facilitates effective knowledge transfer between original and refined models, identifying superior network architectures.

Conclusions:

• The developed constructing method enables the design of significantly more efficient deep learning models.
• The approach not only optimizes network architecture but also enhances model effectiveness and knowledge transfer capabilities.