When Sparse Neural Network Meets Label Noise Learning: A Multistage Learning Framework.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Neural network pruning identifies sparse subnetworks ('winning tickets') for efficiency.
• Existing methods struggle with noisy labels, degrading performance.
• Real-world data often contains label noise, hindering effective pruning.

Purpose of the Study:

• To develop a robust method for identifying winning tickets in the presence of noisy labels.
• To propose a novel two-stream sample selection network (TS3-Net) for this purpose.
• To improve the performance of network pruning techniques on real-world datasets with label noise.

Main Methods:

• Introduced TS3-Net, a two-stream network with sparse and dense subnetworks.
• Employed an iterative training procedure involving weight pruning.
• Utilized a multistage learning framework: warm-up, semisupervised alternate learning, and label refinement.

Main Results:

• TS3-Net effectively identifies winning tickets with noisy labels.
• The proposed method achieves state-of-the-art performance on synthetic and real-world noisy datasets.
• Demonstrated very small memory consumption for label noise learning.

Conclusions:

• TS3-Net offers a robust solution for network pruning under label noise.
• The multistage learning framework progressively enhances sparse subnetwork performance.
• The method shows significant improvements in efficiency and accuracy for noisy datasets.