On cheap entropy-sparsified regression learning.

Area of Science:

• Statistics and Machine Learning
• Computational Science
• Climate Science

Background:

• Regression learning is a fundamental challenge in statistics, machine learning, and deep learning (DL).
• Existing methods often struggle with high-dimensional data and interpretability.
• The complexity of regression tasks necessitates computationally efficient and robust algorithms.

Purpose of the Study:

• To develop a computationally cheap and robust algorithm for regression learning.
• To incorporate the physical principle of entropy maximization into regression analysis.
• To enable efficient application to problems with millions of feature dimensions.

Main Methods:

• Formulated regression as a probabilistic expectation over unknown feature probabilities.
• Incorporated entropy maximization for problem simplification and decomposition.
• Developed the Sparse Probabilistic Approximation for Regression Task Analysis (SPARTAn) algorithm using an efficient second-order numerical solver.

Main Results:

• SPARTAn achieves sublinear cost scaling, enabling analysis of millions of features on a commodity laptop.
• SPARTAn learners provide more predictive, sparse, and physically explainable data descriptions compared to state-of-the-art tools.
• Applied to El Niño Southern Oscillation prediction, SPARTAn identified key roles of ocean temperature variability and timescales.

Conclusions:

• SPARTAn offers a computationally efficient and robust alternative to deep learning for regression tasks.
• The algorithm provides interpretable insights into complex phenomena like climate variability.
• SPARTAn enhances the predictability of key climate drivers by describing feature evolution and surface expression timescales.