Gradient Learning under Tilted Empirical Risk Minimization.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Gradient Learning (GL) is vital for variable selection due to its broad applicability.
• Existing Gradient Learning methods often rely on empirical risk minimization (ERM), which can falter with complex data like non-Gaussian noise.

Purpose of the Study:

• To introduce a novel Gradient Learning model robust to complex data environments.
• To enhance variable selection performance by employing a tilted empirical risk minimization (ERM) criterion.

Main Methods:

• Developed a new Gradient Learning model utilizing a tilted ERM criterion.
• Employed operator approximation techniques for theoretical analysis.
• Proposed a gradient descent algorithm to optimize the learning objective.
• Provided convergence analysis for the proposed algorithm.

Main Results:

• Established theoretical support for the new model from a function approximation perspective.
• Demonstrated the effectiveness of the proposed Gradient Learning approach through simulated experiments.
• Validated performance improvements in the presence of correlated input variables.

Conclusions:

• The proposed tilted ERM-based Gradient Learning model offers improved robustness and performance.
• The method is particularly effective for variable selection in complex data scenarios, including correlated variables and non-Gaussian noise.