Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Multi-input and Multi-variable systems

Multi-input and Multi-variable systems

Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...

Masking and Demasking Agents

Masking and Demasking Agents

EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on the metal...

Barriers to Effective Communication II

Barriers to Effective Communication II

The barriers to effective communication also include cultural barriers, semantic barriers, gender barriers, and time constraints.
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...

Auditory Pathway

Auditory Pathway

Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...

Design Example

Design Example

The innovation of touch-tone telephony revolutionized the telecommunications industry by replacing the traditional rotary dial with a dual-tone multi-frequency (DTMF) signaling system. This system uses a matrix-style keypad with buttons arranged in four rows and three columns, creating 12 distinct signals each assigned to a pair of frequencies. Each button press results in a simultaneous generation of two sinusoidal tones – one from a low-frequency group (697 to 941 Hz) and one from a...

Interference: Path Lengths

Interference: Path Lengths

Consider two sources of sound, that may or may not be in phase, emitting waves at a single frequency, and consider the frequencies to be the same.
Two special sources may be considered when they are in phase. This can be easily achieved by feeding the two sources from the same source. An example would be synchronizing the two speakers by feeding them with the same source, such as the sound waves produced by a tuning fork. This setup ensures that the two sources have the same frequency and are...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same journal

UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation.

IEEE transactions on pattern analysis and machine intelligence·2026

Same journal

Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection.

IEEE transactions on pattern analysis and machine intelligence·2026

Same journal

STPP: Efficient and Progressive Structured Pruning Via Enhanced Sparsification Paradigm.

IEEE transactions on pattern analysis and machine intelligence·2026

Same journal

Incomplete Multimodal Probability Flow Recovery for Emotion Recognition.

IEEE transactions on pattern analysis and machine intelligence·2026

Same journal

MIDAS: Mutual Information Disentanglement With Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis.

IEEE transactions on pattern analysis and machine intelligence·2026

Same journal

How Relation Enrichment Improves Clustering Ensemble Performance: A Second Order Induced Relation View.

IEEE transactions on pattern analysis and machine intelligence·2026

Search research articles

Related Experiment Video

Updated: Jun 3, 2026

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

Multimodal Speaker Diarization.

A Noulas, G Englebienne, B J A Krose

IEEE Transactions on Pattern Analysis and Machine Intelligence

|March 9, 2011

Summary

This study introduces a new audiovisual framework for speaker diarization, outperforming audio-only methods. The dynamic Bayesian network model effectively identifies speakers without needing labeled data.

Area of Science:

Computer Science
Artificial Intelligence
Signal Processing

Background:

Speaker diarization is crucial for understanding audiovisual content.
Existing methods often rely on single modalities (audio or video) or require labeled data.
Robust speaker diarization across diverse recording environments remains a challenge.

Purpose of the Study:

To develop a novel probabilistic framework for speaker diarization by fusing audio and video information.
To create a robust model that does not require labeled training data or make assumptions about recording equipment.
To improve the state-of-the-art in speaker diarization through multimodal analysis.

Main Methods:

A Dynamic Bayesian Network (DBN) framework, extending factorial Hidden Markov Models (fHMM).

Related Experiment Videos

Last Updated: Jun 3, 2026

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

Modeling speakers as multimodal entities generating observations across audio, video, and joint audiovisual spaces.

Parameter acquisition using the Expectation Maximization (EM) algorithm, enabling unsupervised learning.

Main Results:

The proposed multimodal framework significantly outperforms single-modality analysis in speaker diarization.
The model demonstrates improved performance over existing state-of-the-art audio-based speaker diarization techniques.
Successful application to publicly available meeting and news broadcast video datasets.

Conclusions:

The novel audiovisual framework offers a robust and effective solution for speaker diarization.
Multimodal fusion provides superior performance compared to unimodal approaches.
The unsupervised learning capability makes the framework adaptable to various real-world scenarios.