Online Censoring for Large-Scale Regressions with Application to Streaming Big Data

Area of Science:

• Computational Statistics
• Machine Learning
• Optimization

Background:

• Modern linear regression problems are increasingly large-scale, demanding efficient computational solvers.
• Significant data reduction is possible without compromising statistical inference quality or computational feasibility.
• Existing methods may not be suitable for streaming data or decentralized learning scenarios.

Purpose of the Study:

• To introduce methods for identifying and omitting less informative observations in an online, data-adaptive manner.
• To develop efficient solvers for large-scale linear regression problems, particularly for streaming and decentralized data.
• To investigate joint censoring and estimation strategies for centralized, large-scale regression tasks.

Main Methods:

• Utilized first- and second-order stochastic approximation algorithms for sequential maximum-likelihood estimation with streaming data.
• Developed novel online algorithms with closed-form updates and proven convergence guarantees for joint censoring and estimation.
• Investigated thresholding rules to achieve specific censoring patterns and dimensionality reduction.

Main Results:

• Proposed data-adaptive methods effectively identify and omit less informative observations.
• Online algorithms demonstrate provable (non)asymptotic convergence for efficient large-scale linear regression.
• Numerical tests confirm the superiority of proposed methods over data-agnostic random projection techniques.

Conclusions:

• Data-adaptive omission of observations offers a computationally efficient approach to large-scale linear regression.
• The developed online algorithms are suitable for streaming data, censored data, and decentralized learning.
• The proposed methods provide a robust and effective alternative to existing data-agnostic approaches.