Related Experiment Video
Updated: Jul 1, 2025
11:18
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
Published on: March 2, 2015
10.3K
A regularized orthogonal activated inverse-learning neural network for regression and classification with outliers
Zhijun Zhang1, Yating Song1, Tao Chen1
1School of Automation Science and Engineering, South China University of Technology, China.
Summary
A new neural network, regularized orthogonal activated inverse-learning (ROAIL), efficiently reduces outlier impact in regression and classification. The Welsch-ROAIL variant shows strong outlier suppression capabilities with single-step computation.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Data Science
Background:
- Outliers significantly impact regression and classification model performance.
- Traditional methods often require iterative computations, increasing complexity.
- Efficient outlier reduction techniques are crucial for robust data analysis.
Purpose of the Study:
- To introduce a novel regularized orthogonal activated inverse-learning (ROAIL) neural network.
- To develop a computationally efficient method for outlier impact reduction.
- To enhance model robustness in regression and classification tasks.
Main Methods:
- Developed a single-step computation ROAIL neural network.
- Extended Gegenbauer polynomials to a multivariate version.
- Integrated ℓ2 regularization and Welsch loss function, creating ℓ2-ROAIL and W-ROAIL variants.
Main Results:
- ℓ2-ROAIL minimizes empirical and structural risk, improving generalization.
- W-ROAIL enhances robustness by down-weighting outliers using the Welsch function.
- Experiments in regression and classification demonstrated W-ROAIL's strong outlier suppression.
Conclusions:
- ROAIL offers an efficient, single-step approach to neural network weight acquisition.
- W-ROAIL provides superior robustness against outliers compared to ℓ2-ROAIL.
- The proposed ROAIL framework effectively reduces the impact of outliers in machine learning models.
Related Concept Videos
01:08Outliers and Influential Points
4.0K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.0K
01:12What Are Outliers?
3.8K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.8K
01:52Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
01:11Residuals and Least-Squares Property
7.4K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.4K
01:11Regression Analysis
5.7K
Regression analysis is a statistical tool that describes a mathematical relationship between a dependent variable and one or more independent variables.
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
5.7K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K