BioGD: Bio-inspired robust gradient descent.

Area of Science:

• Machine Learning
• Computational Neuroscience
• Optimization Theory

Background:

• State-of-the-art machine learning models struggle to differentiate noise from significant data variations.
• This limitation makes them vulnerable to adversarial perturbations, resulting in confident yet incorrect predictions.
• Brittle robustness properties are highly sensitive to subtle data noise.

Purpose of the Study:

• To investigate if biologically inspired algorithms can improve robustness in machine learning.
• To address the vulnerability of current models to adversarial noise and subtle data variations.

Main Methods:

• Introduction of a novel robust gradient descent algorithm inspired by biological systems' stability and adaptability.
• Incorporation of an open-ended adaptation process with two hyperparameters from the Verhulst population growth equation.
• Penalization of gradient changes' impact on predictions to increase robustness to adversarial noise.

Main Results:

• Empirical evidence on synthetic and experimental datasets confirmed the viability of the bio-inspired gradient descent.
• The proposed technique demonstrated increased robustness to adversarial noise.
• The method showed promise in handling subtle data variations.

Conclusions:

• Bio-inspired gradient descent offers a promising approach to enhance machine learning model robustness.
• The technique effectively mitigates the impact of adversarial perturbations and noise.
• Further research directions are suggested for refining bio-inspired optimization in machine learning.