Related Experiment Video
Updated: May 17, 2026
Frame-by-Frame Video Analysis of Idiosyncratic Reach-to-Grasp Movements in Humans
Published on: January 15, 2018
A machine learning study highlighting the challenges of fidgety movement recognition using vision and inertial
Falco Lentzsch1, Frédéric Li2, Friederike Pagel3
1German Research Center for Artificial Intelligence (DFKI), Luebeck, 23562, Germany. falco.lentzsch@dfki.de.
Insights
Automating the General Movement Assessment for infants is challenging. Deep learning models can learn movement features but struggle to generalize to new subjects for recognizing Fidgety Movements.
Area of Science:
- Neurology
- Developmental Pediatrics
- Machine Learning
Background:
- Infantile movement, particularly Fidgety Movements in infants under 20 weeks, is crucial for early neurological development.
- Absence of Fidgety Movements is linked to neurological disorders like Cerebral Palsy, necessitating early detection.
- The General Movement Assessment (GMA) is a clinical tool for screening Fidgety Movements, but it is time- and resource-intensive.
Purpose of the Study:
- To investigate the use of deep learning for automated Fidgety Movement recognition.
- To develop disentangled feature representations from multimodal data (RGB-D video and IMU) for GMA.
- To address the challenges in generalizing automated GMA models to unseen subjects.
Main Methods:
- Utilized deep learning approaches on RGB-D video and Inertial Measurement Unit (IMU) data from 95 infants.
- Focused on learning disentangled feature representations for Fidgety Movement recognition.
- Evaluated the model's ability to generalize features independently of subject information.
Main Results:
- Deep learning models successfully learned features characterizing infant movement.
- Generalizing these learned features to subjects not included in the training set proved challenging.
- Both vision-based (RGB-D) and sensor-based (IMU) modalities presented unique challenges for accurate Fidgety Movement recognition.
Conclusions:
- While feature learning for infant movement is feasible, robust generalization for automated GMA remains an open problem.
- Specific challenges exist within both vision and sensor data modalities that require further investigation.
- Recommendations are provided for future research in automated Fidgety Movement detection.
Abstract:
Past medical research has shown that infantile movement and early neurological development are closely linked. Fidgety Movements that are reflex-like movement occurring in healthy infants less than 20-week of age have proven to be especially important, as past studies have highlighted that their absence is strongly correlated with the future development of neurological disorders like Cerebral Palsy. To provide a timely intervention, the General Movement Assessment was proposed as a screening medical procedure carried out by clinical personnel specifically trained to recognize Fidgety Movements. Because of its high cost in time and resources, several initiatives to automatize General Movement Assessment using machine learning techniques have been proposed in the literature. However none has managed to emerge as state-of-the-art so far. To investigate this problem, we conducted a study using deep learning approaches to learn disentangled feature representations for the recognition of Fidgety Movements using RGB-D video and Inertial Measurement Unit data acquired from 95 infants (average age: [Formula: see text] weeks). Our results show that while it is possible to learn features that characterize movement independently of subject information, obtaining feature representations that consistently generalize to subjects unseen during training remains challenging. More specifically, we observe that both the vision- and sensor-based modalities have specific challenges to be addressed for the recognition of Fidgety Movements. We discuss them and provide recommendations to help researchers interested in investigating this problem in the future.
