Temporal Contrastive Learning through implicit non-equilibrium memory.

Area of Science:

• Computational neuroscience
• Machine learning
• Statistical physics

Background:

• Backpropagation is a standard neural network training method.
• Local learning methods offer decentralized training for energy-based models.
• Current local methods require explicit memory to contrast behaviors.

Purpose of the Study:

• Introduce Temporal Contrastive Learning (TCL) for energy-based models.
• Develop a method for decentralized training without explicit memory.
• Explore the physical and energetic costs of contrastive learning.

Main Methods:

• Utilize integral feedback for implicit non-equilibrium memory.
• Employ a sawtooth-like temporal protocol for alternating behaviors.
• Analyze learning quality and energy costs through physical dynamics.

Main Results:

• TCL enables contrastive learning using implicit memory.
• Non-equilibrium dissipation enhances learning quality.
• A Landauer-like energy cost for contrastive learning is determined.

Conclusions:

• TCL broadens the applicability of contrastive learning to diverse physical and biological systems.
• The approach offers a pathway for efficient, decentralized learning.
• Understanding the energy cost provides insights into the physical limits of computation.