Comparison of l₁-Norm SVR and Sparse Coding Algorithms for Linear Regression

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Support Vector Regression (SVR) is a powerful function estimation method.
• l1-norm SVR excels at feature selection, especially with redundant features.
• Sparse Coding (SC) is a widely applicable technique with efficient algorithms.

Purpose of the Study:

• To reveal the connection between l1-norm SVR and SC.
• To compare the efficiency of SC and l1-norm SVR algorithms for linear regression.
• To evaluate SC algorithms for designing Radial Basis Function (RBF) neural networks.

Main Methods:

• Comparison of typical l1-norm SVR and SC algorithms for linear regression.
• Application of SC algorithms to RBF neural network design.
• Experimental evaluation on benchmark datasets.

Main Results:

• SC algorithms demonstrate superior efficiency compared to the Newton linear programming algorithm for l1-norm SVR in linear regression.
• SC algorithms show high efficiency in designing RBF neural networks.
• Orthogonal matching pursuit (a SC algorithm) is significantly faster than orthogonal least squares for RBF network design.

Conclusions:

• Sparse Coding algorithms offer a more efficient approach for linear regression and RBF neural network design compared to traditional l1-norm SVR methods.
• Orthogonal matching pursuit presents a highly efficient alternative for RBF network construction.