Cheap robust learning of data anomalies with analytically solvable entropic outlier sparsification

Area of Science:

• Computational statistics
• Machine learning

Background:

• Data anomalies and outliers pose significant challenges in machine learning.
• Existing methods for handling outliers can be computationally expensive or lack robustness.

Purpose of the Study:

• To introduce Entropic Outlier Sparsification (EOS) as an efficient and robust computational strategy for machine learning.
• To analyze the theoretical underpinnings and practical performance of EOS.

Main Methods:

• Derivation of an analytic solution for expected loss minimization with Shannon entropy regularization.
• Development of a closed-form solution for EOS, analyzing its computational complexity.
• Comparison of EOS performance against established methods on synthetic and real-world biomedical datasets.

Main Results:

• EOS imposes computational costs linear to statistics size and independent of data dimension.
• The theoretical framework explains the optimality of Gaussian mixtures in certain data analysis contexts.
• EOS achieved a statistically significant improvement in predicting patient mortality from heart failure data, outperforming common learning tools.

Conclusions:

• Entropic Outlier Sparsification (EOS) provides a computationally efficient and robust method for handling data anomalies in machine learning.
• EOS demonstrates superior performance in critical biomedical applications like mortality prediction.