Confounder-Free Continual Learning via Recursive Feature Normalization.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Confounding variables introduce spurious correlations and bias predictions in machine learning models.
• Existing methods like metadata normalization (MDN) adjust feature distributions but struggle with continual learning.
• Continual learning models face challenges in maintaining invariant feature representations against changing confounders.

Purpose of the Study:

• To develop a novel layer, Recursive MDN (R-MDN), for mitigating confounder influence in deep learning.
• To enable feature representations that are invariant to confounding variables in continual learning settings.
• To improve prediction equity across diverse population groups during both static and continual learning.

Main Methods:

• Introduced the Recursive MDN (R-MDN) layer, adaptable to various deep learning architectures and stages.
• Employed the recursive least squares algorithm for statistical regression to continually update the model's internal state.
• Integrated R-MDN to adjust feature representations based on evolving data and confounding variable distributions.

Main Results:

• Demonstrated R-MDN's effectiveness in promoting equitable predictions across population groups.
• Showcased R-MDN's ability to reduce catastrophic forgetting in continual learning scenarios.
• Validated R-MDN's performance in both static and dynamic learning environments with changing confounders.

Conclusions:

• The R-MDN layer offers a robust solution for handling confounding variables in deep learning, particularly within continual learning frameworks.
• R-MDN enhances model fairness and robustness by ensuring feature invariance to confounders.
• This approach mitigates the negative impact of confounders on model performance and generalization over time.