Geometric Analysis of Uncertainty Sampling for Dense Neural Network Layer.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Model adaptation is crucial for efficient neural network training.
• Active learning strategies aim to reduce labeling effort by selecting informative samples.
• Uncertainty sampling is a common active learning approach.

Purpose of the Study:

• To analyze uncertainty sampling objectives for model adaptation in fully connected neural network layers using information geometry.
• To identify conditions where different uncertainty-based methods perform similarly, particularly in early learning stages.
• To define and evaluate 'risky' samples for adaptation and compare margin-based sampling against other methods.

Main Methods:

• Information geometric and sample-based analysis of active learning uncertainty sampling objectives.
• Definition and identification of 'risky' samples for model adaptation.
• Experimental validation using Meta-Dataset, a few-shot learning benchmark.
• Comparison of uncertainty-based active learning objectives with features from SimpleCNAPS for a fully-connected adaptation layer.

Main Results:

• Conditions for similar performance among uncertainty-based methods are more likely in early learning stages.
• Margin-based sampling preferentially selects 'risky' samples, outperforming other methods as labeled data increases.
• Margin-based uncertainty sampling achieved comparable performance to other methods using fewer labeled samples.

Conclusions:

• Margin-based uncertainty sampling is an efficient strategy for model adaptation in neural networks.
• The geometric analysis provides insights into the behavior of uncertainty sampling methods.
• This approach can significantly reduce the data labeling requirements for few-shot learning tasks.