Sequential linear regression with online standardized data.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Sequential least squares multidimensional linear regression is crucial for data streams.
• Numerical instability and computational time are challenges with traditional methods.
• Existing methods struggle to efficiently process large volumes of incoming data.

Purpose of the Study:

• To develop a robust and efficient method for sequential least squares regression on data streams.
• To address numerical explosion and reduce computation time.
• To propose and analyze novel stochastic approximation processes using online standardized data.

Main Methods:

• Stochastic approximation processes with online standardized data.
• Three distinct processes are defined and analyzed: variable step-size with varying observations, averaged process with constant step-size and varying observations, and a process using all observations.
• Convergence analysis under generalized assumptions.

Main Results:

• The proposed processes demonstrate almost sure convergence.
• Convergence is achieved under more general assumptions than classical methods.
• Empirical comparisons on 11 datasets show the third process (using all observations) typically provides superior results.

Conclusions:

• The developed processes offer improved numerical stability and computational efficiency for data stream regression.
• The process utilizing all observations until the current step is particularly effective.
• The findings advance the field of online regression analysis for big data applications.