Related Experiment Video
Updated: Sep 18, 2025
12:39
A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers
Published on: January 18, 2020
7.8K
Ubiquitous memory augmentation via mobile multimodal embedding system
Dongqi Cai1,2, Shangguang Wang3, Chen Peng1
1State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China.
Nature Communications
|June 19, 2025
Summary
Reminisce is an efficient on-device multimodal embedding system that augments mobile memory. It uses brain-inspired retrieval for high throughput and precise results on resource-constrained devices.
Area of Science:
- Computer Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Multimodal embedding models vectorize reality into a unified space for information retrieval.
- Increasing model capacity leads to high resource consumption, hindering mobile deployment.
- Efficient on-device solutions are needed to augment mobile memory capabilities.
Purpose of the Study:
- To present Reminisce, an efficient on-device multimodal embedding system.
- To enable high-throughput embedding and precise retrieval on resource-constrained mobile devices.
- To address the limitations of current models in terms of resource consumption and speed.
Main Methods:
- Utilizing coarse-grained embeddings for candidate identification, inspired by human memory.
- Employing query-driven fine-grained retrieval for refining search results.
- Implementing algorithm-hardware orchestrated optimizations for enhanced embedding quality and efficiency.
Main Results:
- Reminisce achieves high-quality embedding representation with high throughput.
- The system operates efficiently on resource-constrained mobile devices.
- Negligible memory usage and reduced energy consumption were observed.
Conclusions:
- Reminisce offers an effective solution for on-device multimodal embedding and retrieval.
- The system successfully balances performance and resource efficiency for mobile applications.
- The brain-inspired approach demonstrates potential for future memory augmentation technologies.
Related Concept Videos
01:23System of Memory
6.5K
Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...
6.5K
01:24Implicit Memories
198
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
198
01:23Mnemonic Devices
188
Mnemonic devices are cognitive tools that facilitate memory retention by linking new information to familiar patterns or organizational strategies. These techniques are beneficial for remembering complex or lengthy sets of information by simplifying and structuring them in easily retrievable ways.
Acronyms
Acronyms are created by using the initial letters of a series of words to form a new word or phrase. This approach condenses complex information into a single, memorable entity. For example,...
Acronyms
Acronyms are created by using the initial letters of a series of words to form a new word or phrase. This approach condenses complex information into a single, memorable entity. For example,...
188
01:18Long-Term Memory
266
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
266
01:22Chunking and Rehearsal in Sensory Memory
305
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
305
01:19Understanding Memory
646
Memory is the retention of information or experiences over time, facilitated through three main processes: encoding, storage, and retrieval. Encoding is the process of inputting information into the memory system. For instance, when listening to a lecture, watching a play, reading a book, or having a conversation, the brain is actively encoding information. This initial stage involves transforming sensory input into a form that can be processed and stored by the brain. Various factors, such as...
646