Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Multiaxial Fatigue Life Assessment of Large Welded Flange Shafts: A Continuum Damage Mechanics Approach.

Materials (Basel, Switzerland)·2025

Same author

GR-AttNet: Robotic grasping with lightweight spatial attention mechanism.

PloS one·2025

Same author

Dual-branch differential channel hypergraph convolutional network for human skeleton based action recognition.

PloS one·2025

Same author

MEP-YOLOv5s: Small-Target Detection Model for Unmanned Aerial Vehicle-Captured Images.

Sensors (Basel, Switzerland)·2025

Same author

Highly active and reversible NiPSe<sub>3</sub> anode for sodium-ion batteries: enabling ultrafast sodium storage with exceptional cycling stability.

Journal of colloid and interface science·2025

Same author

Use of BOIvy Optimization Algorithm-Based Machine Learning Models in Predicting the Compressive Strength of Bentonite Plastic Concrete.

Materials (Basel, Switzerland)·2025

Same journal

DSPE-ViT: a lightweight vision transformer with dynamic sparse positional encoding for dense small object detection in UAV imagery.

Frontiers in neurorobotics·2026

Same journal

ST-HONet: Spatio-Temporal Hierarchical Network for long-horizon bimanual visuomotor imitation.

Frontiers in neurorobotics·2026

Same journal

ST-HADP: Spatio-Temporal hierarchical attention diffusion policy for long-horizon generalizable bimanual visuomotor imitation.

Frontiers in neurorobotics·2026

Same journal

EQISP: efficient quantized image signal processing with multi-scale pyramid fusion for resource constrained embodied perception.

Frontiers in neurorobotics·2026

Same journal

Research on embodied agent multimodal perception and real-time path planning algorithms for complex unstructured environments.

Frontiers in neurorobotics·2026

Same journal

NL-YOLOv5: a model with a larger receptive field and the ability to globally acquire features.

Frontiers in neurorobotics·2026

See all related articles

Search research articles

Related Experiment Video

Updated: Jun 7, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

LS-VIT: Vision Transformer for action recognition based on long and short-term temporal difference.

Dong Chen^1,2,3, Peisong Wu^1,3, Mingdong Chen^1,3

¹College of Physics and Electronic Engineering, Nanning Normal University, Nanning, China.

Frontiers in Neurorobotics

|November 15, 2024

Summary

This summary is machine-generated.

This study introduces the Long and Short-term Temporal Difference Vision Transformer (LS-VIT) for efficient 3D video action recognition. The LS-VIT model achieves high accuracy by effectively capturing both short-term and long-term motion details in videos.

Keywords:

Vision Transformer action recognition deep learning motion extraction temporal crossing fusion

More Related Videos

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Published on: November 1, 2024

Related Experiment Videos

Last Updated: Jun 7, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility

Published on: August 9, 2024

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Author Spotlight: Deciphering Electrical Networks Behind Complex Brain Activities and Disorders

Published on: November 1, 2024

Area of Science:

Computer Vision
Artificial Intelligence
Machine Learning

Background:

Transformer models excel in 2D vision but face computational challenges in 3D video tasks like action recognition.
Directly applying temporal transformations to 3D video data increases computational and memory demands due to data patch multiplication and complex self-attention mechanisms.

Purpose of the Study:

To develop an efficient and precise 3D self-attentive model for video action recognition.
To address the computational challenges posed by transformer models in 3D video analysis.

Main Methods:

Introduced the Long and Short-term Temporal Difference Vision Transformer (LS-VIT).
Incorporated short-term motion details by weighting differences across consecutive frames.
Integrated a module for long-term motion understanding using motion excitation and temporal differences from various segments.

Main Results:

LS-VIT achieved high recognition accuracy on multiple benchmarks, including UCF101, HMDB51, and Kinetics-400.
The model effectively models both short-term and long-term motion dynamics in videos.

Conclusions:

LS-VIT demonstrates strong performance in 3D video action recognition.
The model shows potential for further optimization to enhance real-time performance and action prediction capabilities.