Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

373
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
373
Associative Learning01:27

Associative Learning

1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
1.2K
Collisions in Multiple Dimensions: Introduction01:05

Collisions in Multiple Dimensions: Introduction

6.5K
It is far more common for collisions to occur in two dimensions; that is, the initial velocity vectors are neither parallel nor antiparallel to each other. Let's see what complications arise from this. The first idea is that momentum is a vector. Like all vectors, it can be expressed as a sum of perpendicular components (usually, though not always, an x-component and a y-component, and a z-component if necessary). Thus, when the statement of conservation of momentum is written for a...
6.5K
Interactions Between Signaling Pathways01:19

Interactions Between Signaling Pathways

7.2K
Signaling cascades usually lack linearity. Multiple pathways interact and regulate one another, allowing cells to integrate and respond to diverse environmental stimuli.
Convergence and divergence, and cross-talk between signaling pathways
Two distinct signaling pathways can converge on a single functional unit, which may either be a single protein or a complex of proteins. The response is either functionally distinct or synergistic between the two pathways but different from the response...
7.2K
Collisions in Multiple Dimensions: Problem Solving01:06

Collisions in Multiple Dimensions: Problem Solving

5.2K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
5.2K
Support Reactions in Three Dimensions01:27

Support Reactions in Three Dimensions

1.6K
Support reactions in three dimensions help maintain the stability and equilibrium of various structures and systems. These reactions prevent the system from translating and rotating, ensuring the design can withstand external forces and perform its intended function efficiently and safely. Some of the supports providing support reactions in three dimensions are discussed below:
Ball and Socket Joint is one of the supports allowing free rotation about any axis. This freedom of rotation is...
1.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Muscle-to-Bone CX3CL1 Signaling Promotes Skeletal Repair Through CX3CR1+ Osteoprogenitors.

bioRxiv : the preprint server for biology·2026
Same author

Diagnosing Acute Kava Dermopathy: A Case Report of a Characteristic Cutaneous Eruption.

The Medical journal of Australia·2026
Same author

<i>GeSTICS</i>: A Multimodal Corpus for Studying Gesture Synthesis in Two-party Interactions with Contextualized Speech.

Proceedings of the ... ACM International Conference on Intelligent Virtual Agents. IVA (Conference)·2025
Same author

Representation Learning for Interpersonal and Multimodal Behavior Dynamics: A Multiview Extension of Latent Change Score Models.

Proceedings of the ... ACM International Conference on Multimodal Interaction. ICMI (Conference)·2025
Same author

Computational Analysis of Expressive Behavior in Clinical Assessment.

Annual review of clinical psychology·2025
Same author

Dynamic and dyadic relationships between facial behavior, working alliance, and treatment outcomes during depression therapy.

Journal of consulting and clinical psychology·2025
Same journal

Visual Self-Refinement for Autoregressive Models.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
Same journal

README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
Same journal

MedCOD: Enhancing English-to-Spanish Medical Translation of Large Language Models Using Enriched Chain-of-Dictionary Framework.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
Same journal

Large Language Models are In-context Teachers for Knowledge Reasoning.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
Same journal

Using tournaments to calculate AUROC for zero-shot classification with LLMs.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
Same journal

Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing·2026
See all related articles
  1. Home
  3. Beyond Additive Fusion: Learning Non-additive Multimodal Interactions.
  1. Home
  3. Beyond Additive Fusion: Learning Non-additive Multimodal Interactions.

Related Experiment Video

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.4K

Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions.

Torsten Wörtwein1, Lisa B Sheeber2, Nicholas Allen3

  • 1Language Technologies Institute, Carnegie Mellon University.

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing
|December 2, 2025

View abstract on PubMed

Summary
This summary is machine-generated.

Multimodal Residual Optimization (MRO) helps interpret multimodal models by separating unimodal, bimodal, and trimodal interactions. This method quantifies interactions without reducing predictive performance, aligning with human perception.

More Related Videos

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
07:13

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities

Published on: October 27, 2023

1.6K

Related Experiment Videos

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.4K
Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
07:13

Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities

Published on: October 27, 2023

1.6K

Area of Science:

  • Artificial Intelligence
  • Human-Computer Interaction
  • Machine Learning

Background:

  • Multimodal fusion analyzes spoken language with visual and prosodic cues.
  • Current multimodal models lack clarity on interaction learning versus independent modality processing.

Purpose of the Study:

  • To propose Multimodal Residual Optimization (MRO) for separating unimodal, bimodal, and trimodal interactions.
  • To enhance the interpretability of multimodal models by quantifying interaction effects.

Main Methods:

  • MRO prioritizes learning simpler unimodal contributions before complex bimodal and trimodal interactions.
  • Bimodal predictions are trained to correct unimodal prediction residuals, focusing on remaining interactions.

Main Results:

  • MRO effectively separates unimodal, bimodal, and trimodal interactions.
  • The proposed method maintains or improves predictive performance.
  • A human perception study confirmed MRO's learned interactions align with human judgments.

Conclusions:

  • MRO provides a quantifiable and interpretable approach to multimodal interaction analysis.
  • The method enhances understanding of how different modalities contribute to overall performance.
  • MRO offers a principled way to build more transparent and effective multimodal systems.