Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos.

Amir Shahroudy, Tian-Tsong Ng, Yihong Gong

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |April 10, 2017
    PubMed
    Summary

    This study introduces a novel deep learning framework for action recognition using both RGB and depth data. The approach enhances classification accuracy by effectively analyzing complementary cross-modality features.

    Related Concept Videos

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding.

    IEEE transactions on pattern analysis and machine intelligence·2019
    Same author

    Skeleton-Based Online Action Prediction Using Scale Selection Network.

    IEEE transactions on pattern analysis and machine intelligence·2019
    Same author

    Feature Boosting Network For 3D Pose Estimation.

    IEEE transactions on pattern analysis and machine intelligence·2019
    Same author

    Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates.

    IEEE transactions on pattern analysis and machine intelligence·2018
    Same author

    Multimodal Multipart Learning for Action Recognition in Depth Videos.

    IEEE transactions on pattern analysis and machine intelligence·2015
    Same author

    The amplification and evolution of orthologous 22-kDa α-prolamin tandemly arrayed genes in coix, sorghum and maize genomes.

    Plant molecular biology·2010

    Area of Science:

    • Computer Vision
    • Machine Learning
    • Artificial Intelligence

    Background:

    • Single modality action recognition (RGB or depth) has limitations.
    • RGB and depth data offer complementary strengths for action recognition.
    • Analyzing multimodal RGB+D data can improve performance.

    Purpose of the Study:

    • To propose a novel deep autoencoder-based network for shared-specific feature factorization.
    • To develop a structured sparsity learning machine for multimodal signal analysis.
    • To enhance action classification performance by leveraging complementary cross-modality features.

    Main Methods:

    • A deep autoencoder-based shared-specific feature factorization network was developed.
    • Input multimodal signals (RGB+D) were separated into a hierarchy of components.

    Related Experiment Videos

  • A structured sparsity learning machine utilizing mixed norms was proposed for regularization and group selection.

    • Main Results:

    • The proposed framework effectively analyzes cross-modality features.
    • State-of-the-art accuracy was achieved for action classification.
    • Experiments were conducted on five challenging benchmark datasets.

    Conclusions:

    • The developed framework demonstrates the effectiveness of cross-modality feature analysis.
    • The approach successfully integrates RGB and depth data for superior action recognition.
    • The method offers a significant advancement in multimodal action classification.