Competitive Normalized Least-Squares Regression.

Area of Science:

• Machine Learning
• Statistical Learning
• Data Science

Background:

• Online learning is increasingly popular for streaming applications due to low computational needs.
• Online regularized regression is a key area within machine learning.

Purpose of the Study:

• To propose a novel and efficient online regression algorithm.
• To analyze the performance of the proposed algorithm against established methods.

Main Methods:

• Development of the online normalized least-squares (ONLS) algorithm.
• Theoretical analysis comparing ONLS total loss with normalized gradient descent (NGD) and best offline least-squares (LS) predictors.
• Empirical evaluation using real-world data.

Main Results:

• ONLS demonstrates a superior bias-variance tradeoff compared to gradient descent-based LS algorithms.
• ONLS provides better control over feature shrinkage towards the null.
• Empirical studies confirm the high performance of ONLS against state-of-the-art algorithms.

Conclusions:

• ONLS is an efficient and effective algorithm for online regularized regression.
• The algorithm offers significant advantages in bias-variance tradeoff and feature shrinkage.
• ONLS shows strong performance in practical applications with real-world data.