The generalization performance of regularized regression algorithms based on Markov sampling.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Generalization ability of regularized regression algorithms is crucial for predictive modeling.
• Existing research on non-independent and identically distributed (non-i.i.d.) samples has limitations.
• Uniformly ergodic Markov chain (u.e.M.c.) sampling offers a novel approach for analyzing algorithm performance.

Purpose of the Study:

• To investigate the generalization bounds of least square regularized regression (LSRR) and support vector machine regression (SVMR) using u.e.M.c. samples.
• To introduce a new Markov sampling algorithm for generating u.e.M.c. samples in regression tasks.
• To compare the learning performance of LSRR and SVMR under Markov sampling versus random sampling.

Main Methods:

• Derivation of generalization bounds for LSRR and SVMR based on u.e.M.c. samples.
• Development of a novel Markov sampling algorithm inspired by Markov chain Monte Carlo (MCMC) methods.
• Numerical studies evaluating LSRR and SVMR performance using generated u.e.M.c. samples.

Main Results:

• LSRR and SVMR algorithms demonstrate improved generalization performance when trained on u.e.M.c. samples.
• Markov sampling yields significantly smaller mean square errors compared to random sampling.
• Markov sampling also results in reduced variance in the learning performance of LSRR and SVMR.

Conclusions:

• Markov sampling is an effective strategy for enhancing the generalization ability of regularized regression algorithms like LSRR and SVMR.
• The proposed Markov sampling method provides a valuable alternative for handling non-i.i.d. data in machine learning.
• Future research can explore the application of this sampling technique to other machine learning algorithms and datasets.