Learning dynamics of simple perceptrons with non-extensive cost functions

Area of Science:

• Computational Neuroscience
• Machine Learning
• Statistical Physics

Background:

• Gradient descent is a fundamental optimization algorithm in machine learning.
• Traditional gradient descent assumes local updates and extensive statistical properties.
• Human learning exhibits complex behaviors not fully captured by standard models.

Purpose of the Study:

• To propose a novel learning rule for simple perceptrons based on a Tsallis-statistics generalization of gradient descent.
• To investigate the impact of non-extensive cost functions on learning dynamics.
• To compare computational learning results with human learning experiments.

Main Methods:

• A Tsallis-statistics-based generalization of gradient descent dynamics was formulated.
• Langevin equations derived from this generalization were solved numerically.
• Numerical results were compared against a learning curve from a human experiment.

Main Results:

• The generalized learning rule was applied to a simple perceptron.
• Numerical simulations explored different values of the Tsallis index q (extensive and non-extensive cases).
• Excellent agreement was observed between computational results and human learning data for q slightly above unity.

Conclusions:

• Tsallis-statistics-based gradient descent offers a promising learning rule for perceptrons.
• Non-extensivity (non-locality) in computational learning is crucial for mimicking human behavior.
• This approach may lead to more effective artificial intelligence systems that better replicate human cognitive processes.