Machine learning without a processor: Emergent learning in a nonlinear analog network

Area of Science:

• Electronic Engineering
• Machine Learning Hardware
• Analog Computing

Background:

• Standard deep learning requires slow, power-intensive differentiation of large nonlinear networks.
• Existing electronic contrastive local learning networks (CLLNs) are linear, limiting their capabilities for analog machine learning.
• The integration of nonlinear elements into CLLNs for enhanced functionality remains unexplored.

Purpose of the Study:

• To introduce and investigate a nonlinear contrastive local learning network (CLLN).
• To explore the feasibility and utility of incorporating nonlinear elements into electronic learning networks.
• To demonstrate the learning capabilities of nonlinear CLLNs for tasks intractable in linear systems.

Main Methods:

• Development of an analog electronic network using self-adjusting nonlinear resistive elements based on transistors.
• Implementation of a decentralized system architecture for the nonlinear CLLN.
• Testing the network's ability to learn nonlinear tasks, including XOR and nonlinear regression, without external computer assistance.

Main Results:

• The nonlinear CLLN successfully learned tasks, such as XOR and nonlinear regression, that are unachievable with linear systems.
• The decentralized system exhibited error reduction modes (mean, slope, curvature), analogous to spectral bias in artificial neural networks.
• The circuitry demonstrated robustness to damage, rapid retraining (seconds), and ultra-low energy dissipation (picojoules per transistor).

Conclusions:

• Nonlinear CLLNs offer a pathway to fast, efficient, and fault-tolerant analog machine learning hardware.
• The developed system shows significant potential for low-power, high-performance edge computing applications.
• Scalable manufacturability and the study of emergent learning are promising avenues for this technology.