A robust outlier control framework for classification designed with family of homotopy loss function

Area of Science:

• Machine Learning
• Robust Statistics
• Data Mining

Background:

• Traditional loss functions in machine learning can be sensitive to outliers and noise.
• Developing robust models is crucial for reliable performance in real-world applications.
• Existing methods may lack flexibility in adapting to different noise characteristics.

Purpose of the Study:

• To introduce a new, flexible homotopy loss function adaptable to various existing loss functions.
• To enhance the robustness of machine learning models against outliers and adversarial perturbations.
• To propose a novel evaluation standard for measuring and validating model robustness.

Main Methods:

• Developed a tunable homotopy loss function, encompassing L1-norm, logarithmic, Geman-Reynolds, Geman-McClure, and correntropy losses.
• Analyzed the Fisher consistency and robustness of the proposed loss using M-estimation and adversarial perturbation perspectives.
• Integrated the homotopy loss into Least Square Support Vector Machine (LSSVM) and Extreme Learning Machine (ELM) models, solved using a re-weighted least square algorithm.

Main Results:

• The proposed homotopy loss demonstrated superior robustness and anti-interference capabilities against outliers compared to traditional methods.
• The enhanced LSSVM and ELM models exhibited improved performance on datasets with varying noise levels.
• The re-weighted least square algorithm ensured global convergence for the robust models.

Conclusions:

• The novel homotopy loss function provides a flexible and robust alternative for machine learning tasks.
• The proposed robust models significantly improve performance in the presence of noisy data and outliers.
• The new evaluation standard offers a valid measure for assessing model robustness.