A Penalty Approach for Normalizing Feature Distributions to Build Confounder-Free Models

Area of Science:

• Machine Learning in Healthcare
• Medical Image Analysis
• Data Science

Background:

• Clinical applications of machine learning face challenges with explain-ability and confounding factors.
• Confounding variables bias model features by affecting input data and output relationships.
• Existing Meta-Data Normalization (MDN) has limitations due to mini-batch sample size dependency, causing performance oscillations.

Purpose of the Study:

• To extend the Meta-Data Normalization (MDN) method to overcome limitations in handling confounding variables.
• To develop a more robust and adaptable technique for improving machine learning model performance in clinical settings.
• To enhance the explain-ability and accuracy of machine learning models by effectively managing confounding factors.

Main Methods:

• Introduced Penalty-based Meta-Data Normalization (PDMN) by formulating the problem as a bi-level nested optimization.
• Approximated the objective using a penalty method, enabling trainable linear parameters within the MDN layer across all samples.
• Designed PDMN for seamless integration into various architectures, including transformers and recurrent models, without requiring batch-level operations.

Main Results:

• PDMN demonstrated improved model accuracy compared to the original MDN method.
• The new method showed enhanced independence from confounding variables in both synthetic and real-world datasets.
• Successful application in a multi-label, multi-site classification of magnetic resonance images.

Conclusions:

• PDMN offers a significant advancement over MDN for managing confounding variables in machine learning.
• The trainable nature of PDMN parameters allows for broader applicability across diverse model architectures.
• PDMN improves model robustness and reliability for clinical applications, enhancing diagnostic accuracy and interpretability.