Lost in Retraining: Closed-Loop Learning and Model Collapse in Exponential Families.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistical Modeling

Background:

• Closed-loop learning involves models estimating parameters from self-generated data.
• This method is gaining traction for training large neural networks.
• Potential for models to primarily learn from artificial neural network-generated data.

Purpose of the Study:

• To analyze closed-loop learning dynamics for exponential family models.
• To understand parameter evolution and convergence properties.
• To identify methods for mitigating bias amplification in self-supervised learning.

Main Methods:

• Derivation of equations of motion for model parameters.
• Analysis of maximum likelihood estimation (MLE) properties.
• Investigation of maximum a posteriori (MAP) estimation and regularization techniques.

Main Results:

• MLE in closed-loop learning leads to parameter convergence to absorbing states.
• This convergence amplifies initial data biases.
• Bias amplification can be prevented by including ground truth data or using regularization.

Conclusions:

• Closed-loop learning dynamics for exponential families are characterized by parameter evolution.
• Unmitigated MLE can lead to biased models.
• Strategic data inclusion or regularization ensures reliable model training.