Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Event-Based Optical Flow via Transforming Into Motion-Dependent View.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2024
Same author

Event-based asynchronous HDR imaging by temporal incident light modulation.

Optics express·2024
Same author

The Traditional Chinese Medicine Baicalein Potently Inhibits Gastric Cancer Cells.

Journal of Cancer·2016
Same author

Evidence for Inbreeding and Genetic Differentiation among Geographic Populations of the Saprophytic Mushroom Trogia venenata from Southwestern China.

PloS one·2016
Same author

The establishment of species-specific primers for the molecular identification of ten stored-product psocids based on ITS2 rDNA.

Scientific reports·2016
Same author

Evolution and prognosis of breast osteosarcoma: A case report.

Oncology letters·2016

Related Experiment Video

Updated: Jun 12, 2025

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
05:48

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

1.4K

Tackling Event-Based Lip-Reading by Exploring Multigrained Spatiotemporal Clues.

Ganchao Tan, Zengyu Wan, Yang Wang

    IEEE Transactions on Neural Networks and Learning Systems
    |September 17, 2024
    PubMed
    Summary
    This summary is machine-generated.

    This study introduces event cameras for automatic lip-reading (ALR), achieving superior performance with a novel multigrained spatiotemporal feature learning framework. The new method effectively processes event data for accurate word recognition.

    More Related Videos

    Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
    09:27

    Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

    Published on: October 13, 2018

    9.9K
    P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
    06:09

    P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

    Published on: September 8, 2023

    518

    Related Experiment Videos

    Last Updated: Jun 12, 2025

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
    05:48

    Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

    Published on: August 9, 2024

    1.4K
    Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
    09:27

    Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

    Published on: October 13, 2018

    9.9K
    P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
    06:09

    P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

    Published on: September 8, 2023

    518

    Area of Science:

    • Computer Vision
    • Machine Learning
    • Biomedical Engineering

    Background:

    • Automatic lip-reading (ALR) recognizes speech from lip movements.
    • Conventional cameras have limitations for ALR, including lower temporal resolution and higher data redundancy.
    • Event cameras offer advantages like high temporal resolution, reduced data redundancy, and lower power consumption.

    Purpose of the Study:

    • To introduce event cameras as a novel sensing device for automatic lip-reading (ALR).
    • To propose a new multigrained spatiotemporal feature learning framework for processing event data in ALR.
    • To develop and evaluate an effective ALR system using event camera technology.

    Main Methods:

    • Event data is converted into multi-temporal resolution event frames.
    • A multibranch subnetwork learns fine-grained spatial and temporal features from event frames.
    • A temporal aggregation subnetwork models temporal relationships within the event stream.
    • Two new event-based lip-reading datasets (DVS-Lip and DVS-LRW100) were collected.

    Main Results:

    • The proposed framework successfully learns fine-grained spatiotemporal features from microsecond-resolved event data.
    • The system integrates spatially complete and temporally fine features through a multibranch approach.
    • Experimental results show the proposed model outperforms state-of-the-art event-based action recognition and video-based lip-reading models.
    • The collected datasets facilitate further research in event-based lip-reading.

    Conclusions:

    • Event cameras provide significant advantages for automatic lip-reading applications.
    • The novel multigrained spatiotemporal feature learning framework is effective for ALR using event data.
    • The proposed approach demonstrates state-of-the-art performance in event-based lip-reading.