Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Electricity generation from food wastes and microbial community structure in microbial fuel cells.

Bioresource technology·2013
Same author

Plasmon-enhanced photothermoelectric conversion in chemical vapor deposited graphene p-n junctions.

Journal of the American Chemical Society·2013
Same author

Rapid and sensitive detection of vinorelbine in the urine of tumor patients by capillary electrophoresis with tris(2,2'-bipyridyl)ruthenium(II)-based electrochemiluminescence assay.

Analytical sciences : the international journal of the Japan Society for Analytical Chemistry·2013
Same author

Synthesis and biological evaluation of 4-phenoxy-6,7-disubstituted quinolines possessing semicarbazone scaffolds as selective c-Met inhibitors.

Archiv der Pharmazie·2013
Same author

The effects of external electric field: creating non-zero first hyperpolarizability for centrosymmetric benzene and strongly enhancing first hyperpolarizability for non-centrosymmetric edge-modified graphene ribbon H2N-(3,3)ZGNR-NO2.

Journal of molecular modeling·2013
Same author

Msx2 plays a critical role in lens epithelium cell cycle control.

International journal of ophthalmology·2013

Related Experiment Video

Updated: Mar 24, 2026

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
08:15

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Published on: March 28, 2025

1.4K

Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition.

Di Wu, Lionel Pigou, Pieter-Jan Kindermans

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |March 9, 2016
    PubMed
    Summary

    This study introduces Deep Dynamic Neural Networks (DDNN) for multimodal gesture recognition. This novel deep learning approach effectively segments and recognizes gestures using skeleton, depth, and RGB data, achieving high accuracy.

    More Related Videos

    Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
    09:41

    Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

    Published on: April 21, 2023

    2.3K
    Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction
    06:19

    Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction

    Published on: August 16, 2024

    938

    Related Experiment Videos

    Last Updated: Mar 24, 2026

    Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
    08:15

    Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

    Published on: March 28, 2025

    1.4K
    Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
    09:41

    Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

    Published on: April 21, 2023

    2.3K
    Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction
    06:19

    Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction

    Published on: August 16, 2024

    938

    Area of Science:

    • Computer Vision
    • Machine Learning
    • Artificial Intelligence

    Background:

    • Multimodal gesture recognition is challenging due to complex spatio-temporal dynamics.
    • Traditional methods often rely on handcrafted features, limiting adaptability.
    • Deep learning offers potential for automated feature learning in time-series data.

    Purpose of the Study:

    • To propose a novel deep learning framework for multimodal gesture recognition.
    • To develop a method for simultaneous gesture segmentation and recognition.
    • To leverage multimodal data (skeleton, depth, RGB) for enhanced performance.

    Main Methods:

    • A semi-supervised hierarchical dynamic framework using Hidden Markov Models (HMM).
    • Deep Dynamic Neural Networks (DDNN) integrating a Gaussian-Bernouilli Deep Belief Network (DBN) for skeletal data.
    • A 3D Convolutional Neural Network (3DCNN) for processing and fusing depth and RGB image batches.
    • Learning HMM emission probabilities through a data-driven approach.

    Main Results:

    • Achieved a Jaccard index score of 0.81 on the ChaLearn LAP gesture spotting challenge.
    • Demonstrated performance comparable to state-of-the-art hand-tuned feature-based and learning-based methods.
    • Successfully learned high-level spatio-temporal representations from multimodal inputs.

    Conclusions:

    • Deep Dynamic Neural Networks (DDNN) provide an effective purely data-driven approach for multimodal gesture recognition.
    • This method advances the application of deep learning in analyzing complex time-series data.
    • Opens avenues for further research in deep learning for multimodal sensor fusion.