Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Absolute Motion Analysis- General Plane Motion01:24

Absolute Motion Analysis- General Plane Motion

522
Visualize a drone, with its propellers spinning rapidly, hovering mid-air. The fascinating movements and operations of this drone can be comprehended by applying the principle of general plane motion.
As the drone's propellers rotate, an upward force is generated that counteracts the force of gravity, enabling the drone to lift off from the ground. This initial movement of the drone is along a straight path, representing a form of translational motion. In this phase, every point on the...
522
Relative Motion Analysis - Velocity01:24

Relative Motion Analysis - Velocity

679
A stroke engine has a slider-crank mechanism that converts rotational motion from the crank into linear motion of the slider or vice versa. This mechanism consists of three main parts: the crank, the connecting rod, and the slider.
When an external force is exerted, it sets the crank into a rotational movement. This, in turn, instigates the motion of the connecting rod, leading to what is referred to as a general plane motion. This process involves two key points - point A on the connecting rod...
679
Relative Motion Analysis using Rotating Axes01:25

Relative Motion Analysis using Rotating Axes

870
Consider a component AB undergoing a linear motion. Along with a linear motion, point B also rotates around point A. To comprehend this complex movement, position vectors for both points A and B are established using a stationary reference frame.
However, to express the relative position of point B relative to point A, an additional frame of reference, denoted as x'y', is necessary. This additional frame not only translates but also rotates relative to the fixed frame, making it...
870
  1. Home
  3. Research Domains
  5. Information And Computing Sciences
  7. Computer Vision And Multimedia Computation
  9. Video Processing
  11. Motionllm: Understanding Human Behaviors From Human Motions And Videos

MotionLLM: Understanding Human Behaviors from Human Motions and Videos

Ling-Hao Chen, Shunlin Lu, Ailing Zeng

    IEEE Transactions on Pattern Analysis and Machine Intelligence
    |October 31, 2025

    Related Experiment Videos

    Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans
    10:51

    Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans

    Published on: January 15, 2018

    8.8K
    Decoding Natural Behavior from Neuroethological Embedding
    08:00

    Decoding Natural Behavior from Neuroethological Embedding

    Published on: October 3, 2025

    575
    Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
    08:25

    Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment

    Published on: May 7, 2019

    9.5K

    View abstract on PubMed

    Summary
    This summary is machine-generated.

    This study introduces MotionLLM, a framework for human behavior understanding using both video and motion data. It enhances spatial-temporal comprehension and reasoning by jointly modeling multi-modal inputs.

    Area of Science:

    • Computer Vision
    • Artificial Intelligence
    • Human Behavior Analysis

    Background:

    • Current Large Language Models (LLMs) often focus on single modalities (video or motion) for human behavior understanding.
    • Effective human behavior analysis requires integrating information from both visual (video) and kinematic (motion) data to capture detailed body dynamics.

    Purpose of the Study:

    • To develop a novel framework, MotionLLM, for comprehensive human behavior understanding, including captioning and reasoning.
    • To propose a unified training strategy that leverages complementary data from video-text and motion-text pairs.
    • To introduce a new dataset (MoVid) and benchmark (MoVid-Bench) for evaluating multi-modality human behavior understanding.

    Main Methods:

    • Developed MotionLLM, a framework integrating video and motion data for human behavior analysis.

    Related Experiment Videos

    Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans
    10:51

    Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans

    Published on: January 15, 2018

    8.8K
    Decoding Natural Behavior from Neuroethological Embedding
    08:00

    Decoding Natural Behavior from Neuroethological Embedding

    Published on: October 3, 2025

    575
    Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
    08:25

    Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment

    Published on: May 7, 2019

    9.5K
  • Implemented a unified video-motion training strategy using coarse video-text and fine-grained motion-text data.
  • Collected the MoVid dataset and created the MoVid-Bench for robust evaluation.

    • Main Results:

    • MotionLLM demonstrates superior performance in human motion captioning.
    • The framework shows enhanced capabilities in spatial-temporal comprehension.
    • MotionLLM exhibits improved reasoning abilities in human behavior understanding tasks.

    Conclusions:

    • Jointly modeling video and motion data is crucial for nuanced human behavior understanding.
    • MotionLLM offers an effective approach for multi-modality human behavior analysis.
    • The MoVid dataset and MoVid-Bench provide valuable resources for future research.