Related Experiment Video
Updated: Aug 12, 2025
08:35
An Operant Intra-/Extra-dimensional Set-shift Task for Mice
Published on: January 22, 2016
12.3K
On separating long- and short-term memories in hyperdimensional computing
Jeffrey L Teeters1, Denis Kleyko1,2, Pentti Kanerva1
1Redwood Center for Theoretical Neuroscience, University of California, Berkeley, Berkeley, CA, United States.
Frontiers in Neuroscience
|January 26, 2023
Summary
Associative memory offers a more efficient method for storing numerous key-value pairs compared to superposition vectors, requiring less memory and computation for reliable data retrieval. This approach enhances memory capacity without increasing vector width.
Area of Science:
- Computational Neuroscience
- Machine Learning
- Information Theory
Background:
- High-dimensional, fixed-width vectors enable information distribution via superposition, encoding key-value pairs through component-wise multiplication and addition.
- Superposition vectors act as approximate memories, with exact retrieval from a codebook (item memory) requiring vectors of the same width.
- Increasing memory capacity necessitates wider vectors, posing scalability challenges for superposition-based storage.
Purpose of the Study:
- To investigate the efficiency of associative memory versus superposition vectors for storing a large number of key-value pairs.
- To demonstrate that associative memory offers advantages in memory and computation for reliable storage.
- To explore the potential of associative memory to augment superposition vector-based systems.
Main Methods:
- Comparing the memory and computational requirements of associative memory and superposition vector approaches for storing over 1,000 key-value pairs.
- Experimentally validating the advantages of associative memory using the storage of large finite-state automata.
- Analyzing the reliability of storage and retrieval for both memory paradigms.
Main Results:
- Associative memory requires less memory and computation than superposition vectors for storing a large number of key-value pairs with comparable reliability.
- The number of storage locations in associative memory can be increased without increasing vector width, overcoming a key limitation of superposition vectors.
- Experimental results with finite-state automata confirm the practical advantages of associative memory.
Conclusions:
- Associative memory presents a more scalable and efficient solution for large-scale key-value pair storage compared to superposition vectors.
- Associative memory can augment superposition vector systems, analogous to the relationship between human long-term and working memory.
- The findings suggest potential applications in modeling brain-like memory and state-dependent behaviors.
Related Concept Videos
01:23Storage
117
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
117
01:23System of Memory
6.3K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
6.3K
01:18Long-Term Memory
240
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
240
01:22Chunking and Rehearsal in Sensory Memory
267
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
267
01:24Working Memory
409
Working memory refers to a combination of components, including short-term memory and attention, that allow an individual to hold information temporarily as we perform cognitive tasks. It is an essential cognitive function that enables the execution of complex tasks such as problem-solving, comprehension, and reasoning. Unlike short-term memory, which simply involves the storage of information for a brief period, working memory involves the active manipulation and processing of this...
409
01:26Higher Mental Functions of Brain: Learning and Memory
927
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
927