Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Sensory Modalities

Sensory Modalities

Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...

Protein Networks

Protein Networks

An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...

Network Covalent Solids

Network Covalent Solids

Network covalent solids contain a three-dimensional network of covalently bonded atoms as found in the crystal structures of nonmetals like diamond, graphite, silicon, and some covalent compounds, such as silicon dioxide (sand) and silicon carbide (carborundum, the abrasive on sandpaper). Many minerals have networks of covalent bonds.
To break or to melt a covalent network solid, covalent bonds must be broken. Because covalent bonds are relatively strong, covalent network solids are typically...

System of Memory

System of Memory

Memory is categorized into three major systems: sensory memory, short-term memory (STM), and long-term memory (LTM). These systems differ in their capacity and the duration for which they can hold information. Sensory memory captures raw sensory input from the environment, holding it for just a few seconds or less. For example, on hearing a brief, loud sound, like a car horn honking, the sound seems to linger in the mind for a moment even after it stops. This is an instance of sensory memory...

Working Memory

Working Memory

Working memory refers to a combination of components, including short-term memory and attention, that allow an individual to hold information temporarily as we perform cognitive tasks. It is an essential cognitive function that enables the execution of complex tasks such as problem-solving, comprehension, and reasoning. Unlike short-term memory, which simply involves the storage of information for a brief period, working memory involves the active manipulation and processing of this...

Long-Term Memory

Long-Term Memory

Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Flexible MOF nanosheet based membranes for high-flux antifouling oil/water separation and reverse cross-flow cleaning.

Journal of hazardous materials·2026

Same author

Dual-mode switchable and reconfigurable Van der Waals phototransistor for multi-state image encryption.

Light, science & applications·2026

Same author

Ego-R1: Agentic Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning.

IEEE transactions on pattern analysis and machine intelligence·2026

Same author

Contrasting effects of rice-crayfish co-culture on insect herbivores at field and landscape scales.

Pest management science·2026

Same author

Elastic Staining Features in Pulmonary Non-Mucinous Adenocarcinoma and Associated In Situ Lesions: Diagnostic Implications.

International journal of surgical pathology·2026

Same author

Complete chloroplast genome characterization and phylogenetic analysis of two natural caffeine-free <i>Camellia yungkiangensis</i> H. T. Chang accessions.

Frontiers in plant science·2026

Search research articles

Related Experiment Video

Updated: Jan 21, 2026

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

Published on: December 24, 2015

Holistic Multi-modal Memory Network for Movie Question Answering.

Anran Wang, Anh Tuan Luu, Chuan-Sheng Foo

IEEE Transactions on Image Processing : a Publication of the IEEE Signal Processing Society

|August 10, 2019

Summary

This study introduces a Holistic Multi-modal Memory Network (HMMN) for question answering. The HMMN framework improves context retrieval by considering all interactions between multi-modal data, questions, and answer choices.

More Related Videos

Author Spotlight: Exploring Dynamic Neural Changes Associated with Religious Chanting

Author Spotlight: Exploring Dynamic Neural Changes Associated with Religious Chanting

Published on: May 31, 2024

Multi-Modal Home Sleep Monitoring in Older Adults

Multi-Modal Home Sleep Monitoring in Older Adults

Published on: January 26, 2019

Related Experiment Videos

Last Updated: Jan 21, 2026

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

Published on: December 24, 2015

Author Spotlight: Exploring Dynamic Neural Changes Associated with Religious Chanting

Author Spotlight: Exploring Dynamic Neural Changes Associated with Religious Chanting

Published on: May 31, 2024

Multi-Modal Home Sleep Monitoring in Older Adults

Multi-Modal Home Sleep Monitoring in Older Adults

Published on: January 26, 2019

Area of Science:

Artificial Intelligence
Natural Language Processing
Computer Vision

Background:

Multi-modal question answering (QA) is complex due to diverse data integration.
Current methods inadequately capture interactions between data sources in attention mechanisms.

Purpose of the Study:

To propose a novel framework, the Holistic Multi-modal Memory Network (HMMN), for enhanced multi-modal QA.
To improve context retrieval by considering all input source interactions and answer choices.

Main Methods:

Developed the Holistic Multi-modal Memory Network (HMMN) framework.
Implemented a holistic attention mechanism to integrate multi-modal context, questions, and answer choices at each processing hop.
Incorporated answer choices into the context retrieval stage.

Main Results:

The HMMN framework demonstrated effectiveness on the MovieQA and TVQA datasets.
Ablation studies confirmed the significance of holistic reasoning and specific attention strategies.

Conclusions:

The HMMN framework offers a more informative approach to context retrieval for multi-modal QA.
Holistic integration of diverse data sources and answer choices is crucial for advancing multi-modal QA systems.