Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Mode of action analysis for induction of mouse lung tumors by permethrin: Involvement of CYP2F2 enzyme and human relevancy.

Regulatory toxicology and pharmacology : RTP·2025
Same author

Tail Bounds on the Runtime of Categorical Compact Genetic Algorithm.

Evolutionary computation·2024
Same author

ATNAS: Automatic Termination for Neural Architecture Search.

Neural networks : the official journal of the International Neural Network Society·2023
Same author

Malignant oligoastrocytoma in the spinal cord of a cat.

The Journal of veterinary medical science·2022
Same author

Deep neural network for the determination of transformed foci in Bhas 42 cell transformation assay.

Scientific reports·2021
Same author

Latent 3D Volume for Joint Depth Estimation and Semantic Segmentation from a Single Image.

Sensors (Basel, Switzerland)·2020

Related Experiment Video

Updated: Sep 25, 2025

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
08:15

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Published on: March 28, 2025

780

Evaluation of text-to-gesture generation model using convolutional neural network.

Eiichi Asakawa1, Naoshi Kaneko2, Dai Hasegawa3

  • 1Yokohama National University, 79-7 Tokiwadai, Hodogaya-ku, Yokohama, Kanagawa 240-8501, Japan.

Neural Networks : the Official Journal of the International Neural Network Society
|April 26, 2022
PubMed
Summary

This study developed a deep learning model for generating conversational gestures from spoken text. The model shows comparable or superior performance to existing methods and demonstrates successful transferability between speakers.

Keywords:
Convolutional neural networkDeep learningGesture generationSpoken textTransformer architecture

More Related Videos

Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction
06:19

Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction

Published on: August 16, 2024

566
Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication
07:18

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication

Published on: January 26, 2024

989

Related Experiment Videos

Last Updated: Sep 25, 2025

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
08:15

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision

Published on: March 28, 2025

780
Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction
06:19

Integration of Animal Behavioral Assessment and Convolutional Neural Network to Study Wasabi-Alcohol Taste-Smell Interaction

Published on: August 16, 2024

566
Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication
07:18

Author Spotlight: Deciphering the Cognitive and Neural Mechanisms of Gesture in Communication

Published on: January 26, 2024

989

Area of Science:

  • Artificial Intelligence
  • Human-Computer Interaction
  • Robotics

Background:

  • Conversational gestures are vital for natural human-robot and human-virtual agent interaction.
  • Data-driven methods, including deep learning, offer promising avenues for automated gesture generation.
  • Existing speech-to-gesture models require further optimization for quality and transferability.

Purpose of the Study:

  • To experimentally analyze a deep learning-based text-to-gesture generation model.
  • To evaluate the model's performance against existing speech-to-gesture systems.
  • To investigate factors influencing text-to-gesture model performance and transferability.

Main Methods:

  • Utilized a convolutional neural network (CNN) for gesture generation from spoken text.
  • Prepared a dataset by augmenting existing gesture motion data with text information.
  • Trained and evaluated models using speaker-specific data and conducted user perceptual studies.

Main Results:

  • The proposed deep learning model achieved performance comparable or superior to existing speech-to-gesture models.
  • Data cleansing and loss function selection were found to be important factors.
  • The model demonstrated successful transferability between different speakers.
  • Transformer architecture also yielded good quality gestures.

Conclusions:

  • The developed deep learning model effectively generates conversational gestures from spoken text.
  • The model's performance and transferability highlight its potential for enhancing human-agent interactions.
  • Further research into data optimization and architectural choices can improve text-to-gesture generation.