A Connection Between Pattern Classification by Machine Learning and Statistical Inference With the General Linear...

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• The general linear model (GLM) is a cornerstone of frequentist statistical testing.
• Machine learning (MLE) inference offers alternative approaches to parameter estimation and prediction.
• Bridging these methodologies can enhance understanding and performance in data analysis.

Purpose of the Study:

• To derive and illustrate a connection between GLM with frequentist testing and MLE inference.
• To introduce a refined predictive statistical test based on linear Support Vector Machines (SVM).
• To compare classification performance and error estimation between different modeling approaches.

Main Methods:

• Expressing GLM parameter estimation as a Linear Regression Model (LRM) of an indicator matrix.
• Utilizing a linear Support Vector Machine (SVM) for enhanced predictive statistical testing.
• Employing permutation analysis, residual scores, and upper bounds for MLE-based inference.
• Comparing parameter estimations and classification performance using experimental and real data.

Main Results:

• A novel connection is established between GLM and LRM, linked by a normalization value.
• The linear SVM, derived from MLE, provides a more refined predictive test.
• Parameter estimations from GLM and LRM yield different classification performances in the inverse problem.
• MLE-based inference, including model-free estimators, demonstrates an efficient trade-off between Type I errors and statistical power.

Conclusions:

• The derived connection between GLM and LRM offers new perspectives on statistical modeling.
• Linear SVM provides a powerful tool for improved error estimation and classification.
• MLE-based inference shows promise for balancing statistical power and error control in real-world applications.