Experimental Design for Overparameterized Learning With Application to Single Shot Deep Active Learning.

Area of Science:

• Machine Learning
• Deep Learning
• Data Science

Background:

• Modern machine learning models require vast labeled datasets for optimal performance.
• Access to large labeled datasets is often a significant bottleneck due to cost and availability.
• Classical optimal experimental design methods are insufficient for overparameterized models like deep neural networks.

Purpose of the Study:

• To develop a novel data selection strategy applicable to overparameterized regression and interpolation.
• To address the limitations of classical experimental design in the context of modern machine learning.
• To propose an effective algorithm for single-shot deep active learning.

Main Methods:

• The study proposes a new design strategy tailored for overparameterized models.
• The method is demonstrated within the domain of deep learning.
• A novel algorithm for single-shot deep active learning is introduced.

Main Results:

• Classical experimental design, focused on variance reduction, is inadequate for overparameterized models where bias or mixed error dominates.
• The proposed design strategy is effective for overparameterized regression and interpolation tasks.
• The new algorithm facilitates efficient data curation for deep active learning.

Conclusions:

• A new paradigm for optimal experimental design is necessary for overparameterized machine learning models.
• The proposed strategy and algorithm offer a viable solution for data selection in deep active learning.
• This work bridges the gap between classical experimental design theory and modern deep learning practices.