Related Experiment Videos
Hashtag2Action: Data Engineering and Self-Supervised Pre-Training for Action Recognition in Short-Form Videos
Yang Qian1, Ali Kargarandehkordi1, Yinan Sun1
1University of Hawai'i at Mānoa.
... IEEE International Conference on Computer Vision Workshops. IEEE International Conference on Computer Vision
|July 29, 2026
Summary
We developed Hashtag2Action (H2A), a pipeline to create large action recognition datasets from social media videos. This method enables competitive model performance without manual annotation, advancing video understanding research.
Area of Science:
- Computer Vision
- Machine Learning
- Data Science
Background:
- Short-form social media videos are valuable behavioral data sources.
- Weak labels and privacy concerns limit their direct use in action recognition.
Purpose of the Study:
- To introduce an end-to-end pipeline, Hashtag2Action (H2A), for creating large-scale, ethically curated action recognition datasets from social media.
- To leverage weakly labeled data for training robust action recognition models.
Main Methods:
- Developed an adaptive hashtag mining, metadata filtering, and vision-based frame validation pipeline.
- Assembled a dataset of 283,582 clips across 386 action categories.
- Pre-trained a VideoMAE V2 backbone using self-supervised learning on the curated dataset.
Main Results:
- Achieved high top-1 accuracy on benchmark datasets: 99.1% (UCF101), 86.1% (HMDB51), 85.5% (Kinetics-400), and 74.3% (SSv2).
- Demonstrated competitive performance using only 20% of the original pre-training data.
- Showcased the effectiveness of weakly labeled, ethically curated short-form videos.
Conclusions:
- Weakly labeled short-form videos, when carefully curated, can support strong downstream action recognition performance.
- The Hashtag2Action pipeline offers a scalable and ethical approach to dataset creation.
- Publicly released pre-trained weights and metadata facilitate reproducible research in video understanding.
Related Concept Videos
01:28Automatic Processing and Automatic Social Behavior
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
01:06Fixed Action Patterns
A fixed action pattern (FAP) is a specific, hard-wired sequence of behaviors that occurs in response to an external stimulus, called a sign stimulus. The behavior is “fixed” because it is essentially unchangeable—proceeding similarly across individuals of a species every time it occurs.