Robust high-dimensional memory-augmented neural networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Neuroscience

Background:

• Traditional neural networks (NNs) require extensive data and slow training, limiting adaptability.
• Memory-augmented neural networks (MANNs) use explicit memory but face bottlenecks with conventional computer architectures.
• Soft read/write operations in MANNs are inefficient on von Neumann architectures.

Purpose of the Study:

• To propose a robust architecture overcoming the MANN memory access bottleneck.
• To enable efficient in-memory computation on high-dimensional (HD) vectors.
• To achieve near 32-bit software-equivalent accuracy in analog computation.

Main Methods:

• Developed a computational memory unit performing analog in-memory computation on HD vectors.
• Employed a content-based attention mechanism using uncorrelated HD vectors.
• Utilized phase-change memory devices for experimental validation.

Main Results:

• Demonstrated efficacy on few-shot image classification tasks using the Omniglot dataset.
• Achieved high accuracy comparable to 32-bit software.
• Successfully implemented on over 256,000 phase-change memory devices.

Conclusions:

• The proposed architecture merges deep NN representations with HD computing.
• Paves the way for robust vector-symbolic manipulations.
• Enables advancements in reasoning, data fusion, and compression.