An in-depth analysis of data reduction methods for sustainable deep learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models achieve high performance on complex tasks due to advanced models, large datasets, and computational power.
• Increased model complexity and data volume lead to significant energy consumption and storage challenges in training and inference.
• Data reduction offers a potential solution to mitigate the environmental and computational costs associated with deep learning.

Purpose of the Study:

• To present and evaluate multiple data reduction methods for deep learning training datasets.
• To introduce a novel topological representativeness metric for assessing data reduction effectiveness.
• To investigate the impact of data reduction on energy consumption, model performance, and data representativeness.

Main Methods:

• Developed and implemented eight distinct data reduction techniques for tabular datasets.
• Created a Python package to facilitate the application of these data reduction methods.
• Introduced a topology-based metric to quantify the representativeness of reduced datasets compared to the original.
• Extended data reduction methodologies for application to image datasets in object detection tasks.

Main Results:

• Experimentally compared the performance of eight data reduction methods.
• Evaluated the trade-offs between dataset size, representativeness, energy consumption, and model predictive performance.
• Demonstrated the effectiveness of data reduction in potentially lowering energy usage during deep learning model training.

Conclusions:

• Data reduction strategies can effectively address the efficiency challenges in deep learning.
• The proposed methods and metric provide valuable tools for optimizing deep learning workflows.
• Further research can explore the broader applicability and impact of these techniques across various deep learning domains.