HasLoss: a novel Hassanat distance-based loss functions for binary classification.

Area of Science:

• Machine Learning
• Computer Science
• Optimization Theory

Background:

• Loss functions are crucial for training machine learning models, especially neural networks in classification.
• Existing loss functions may lack robustness to outliers and noise, impacting model performance.

Purpose of the Study:

• To establish a theoretical framework for distance-based loss functions using Hassanat distance for binary classification.
• To develop and empirically validate novel Hassanat-based loss function variants.

Main Methods:

• Adapted Hassanat distance for binary classification to create a theoretical framework for distance-based loss functions.
• Conducted gradient analysis to prove bounded gradients and finite Lipschitz constants for Hassanat losses.
• Empirically evaluated six Hassanat loss variants on synthetic and nine real-world datasets, comparing against Binary Cross-Entropy (BCE), Focal Loss, Mean Squared Error (MSE), and L1 Loss.

Main Results:

• Hassanat-based losses demonstrated competitive performance, with comparable or improved calibration, convergence speed, precision, recall, F1-score, and AUC.
• The proposed losses showed notable robustness to outliers and noise.
• Cohen's d analysis indicated some Hassanat variants have a larger practical effect size than BCE.

Conclusions:

• Established theoretical foundations and empirical validation for distance-based Hassanat loss functions.
• Bounded gradients and finite Lipschitz constants provide optimization guarantees and explain observed robustness.
• The framework enables systematic development of robust loss functions for specific applications.