A Probabilistic Formulation for Meta-Weight-Net.

Area of Science:

• Machine Learning
• Deep Learning
• Artificial Intelligence

Background:

• Deep neural networks (DNNs) excel in supervised learning but are vulnerable to training set biases such as label noise and class imbalance.
• Existing example reweighting methods often require manual hyperparameter tuning.
• Meta-Weight-Net (MW-Net) automates weighting function learning but lacks statistical grounding.

Purpose of the Study:

• To propose a probabilistic formulation of MW-Net (PMW-Net) for enhanced robustness in DNNs.
• To introduce statistical rigor and flexibility to meta-learning-based example reweighting.
• To improve DNN performance on biased datasets.

Main Methods:

• Developed probabilistic MW-Net (PMW-Net) by treating the weighting function probabilistically.
• Incorporated additional randomness and controlled weighting function flexibility during learning.
• Extended PMW-Net to fully Bayesian models for further robustness improvements.

Main Results:

• PMW-Net demonstrated improved performance compared to the original MW-Net on synthetic and real datasets.
• The probabilistic approach enhanced the robustness of DNNs against training data biases.
• The method offers greater flexibility in controlling weighting functions.

Conclusions:

• Probabilistic MW-Net (PMW-Net) provides a statistically sound and more flexible alternative to deterministic MW-Net.
• PMW-Net enhances DNN robustness and performance, particularly on datasets with label noise or class imbalance.
• The framework can be extended to Bayesian models for further improvements in robustness.