Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

208
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
208
Chunking and Rehearsal in Sensory Memory01:22

Chunking and Rehearsal in Sensory Memory

202
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
202

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Adaptive regularized spectral reduction for stabilizing ill-conditioned bone-conducted speech signals.

PeerJ. Computer science·2025
Same author

TranStutter: A Convolution-Free Transformer-Based Deep Learning Method to Classify Stuttered Speech Using 2D Mel-Spectrogram Visualization and Attention-Based Feature Representation.

Sensors (Basel, Switzerland)·2023
Same author

Data augmentation and deep neural networks for the classification of Pakistani racial speakers recognition.

PeerJ. Computer science·2022
Same author

Multi-label emotion classification of Urdu tweets.

PeerJ. Computer science·2022
Same author

Comparative analysis on Facebook post interaction using DNN, ELM and LSTM.

PloS one·2019
Same author

A Database as a Service for the Healthcare System to Store Physiological Signal Data.

PloS one·2016

Related Experiment Video

Updated: Jun 27, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

443

Real-time multilingual speech recognition and speaker diarization system based on Whisper segmentation.

Ke-Ming Lyu1, Ren-Yuan Lyu1, Hsien-Tsung Chang1,2,3

  • 1Computer Science and Information Engineering, Chang Gung University, Taoyuan, Taiwan.

Peerj. Computer Science
|April 25, 2024
PubMed
Summary
This summary is machine-generated.

This study introduces a real-time multilingual speech system using OpenAI's Whisper model for accurate speaker diarization (SD) and automatic speech recognition (ASR), even with accents and speaker changes.

Keywords:
Automatic speech recognitionIncremental clusteringReal-time systemSpeaker diarization

More Related Videos

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

1.5K
Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

5.6K

Related Experiment Videos

Last Updated: Jun 27, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

443
Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

1.5K
Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

5.6K

Area of Science:

  • Speech Processing
  • Artificial Intelligence
  • Natural Language Processing

Background:

  • Traditional speech recognition and speaker diarization systems struggle with dynamic, multilingual environments, especially with accents and frequent speaker changes.
  • Accurate processing of Mandarin speech with Taiwanese accents and managing speaker turns are significant challenges in current systems.

Purpose of the Study:

  • To develop a cutting-edge, real-time multilingual speech recognition and speaker diarization system.
  • To enhance performance in complex multispeaker scenarios, focusing on Mandarin with Taiwanese accents and rapid speaker transitions.

Main Methods:

  • Leveraged OpenAI's Whisper model for core speech recognition capabilities.
  • Integrated advanced speaker diarization techniques, including efficient output handling and speaker embedding technology for real-time application.
  • Optimized the system for dynamic, multispeaker environments with frequent speaker switches.

Main Results:

  • Achieved a promising word diarization error rate (WDER) of 6.96% overall.
  • Demonstrated strong performance in two-speaker (2.68% WDER) and three-speaker (11.65% WDER) scenarios.
  • Real-time performance comparable to non-real-time baseline models.

Conclusions:

  • The developed system represents a significant advancement in real-time multilingual speech processing.
  • The system effectively handles complex conversational dynamics, including diverse accents and multiple speakers.
  • The research validates the efficacy of integrating advanced ASR and SD techniques for real-world applications.