Generalized robust loss functions for machine learning.

Area of Science:

• Machine Learning
• Robust Statistics

Background:

• Loss functions are crucial in machine learning, but noise can degrade performance.
• Existing robust loss functions lack unified frameworks and often neglect normal data points.
• Current methods offer limited performance gains in handling noisy data.

Purpose of the Study:

• To introduce a general, unified framework for robust loss functions in machine learning (RML).
• To develop a method that adaptively flattens unbounded loss functions, considering both noise and normal points.
• To improve the performance and robustness of machine learning models in the presence of noise.

Main Methods:

• Developed the Robust Loss functions for Machine Learning (RML) framework with scale and shape parameters.
• Applied RML to flatten Hinge and Square loss functions, creating FHSVM and FLSSVM classifiers.
• Utilized the stochastic variance reduced gradient (SVRG) optimization approach for the proposed models.

Main Results:

• FHSVM and FLSSVM classifiers demonstrated superior performance in distinguishing data types.
• Both models consistently achieved top rankings in extensive experiments.
• FHSVM attained an average accuracy of 81.07% (F-score 73.25%), and FLSSVM achieved 81.54% (F-score 75.71%).

Conclusions:

• The RML framework provides a unified and effective approach to robust loss function design.
• The proposed FHSVM and FLSSVM classifiers significantly improve upon existing methods for noisy datasets.
• The adaptive flattening mechanism offers a promising direction for future research in robust machine learning.