Related Experiment Videos
Intelligent temporal subsampling of American Sign Language using event boundaries
D H Parish1, G Sperling, M S Landy
1Department of Psychology, New York University, New York 10003.
Summary
Selecting key frames at event boundaries improves American Sign Language (ASL) video intelligibility. This method is more effective than regular frame selection for dynamic and static ASL recognition.
Area of Science:
- Computer Vision
- Human-Computer Interaction
- Linguistics
Background:
- Representing video sequences efficiently is crucial for applications like sign language recognition.
- Traditional methods often select frames at regular intervals, potentially missing critical information.
Purpose of the Study:
- To investigate if representing American Sign Language (ASL) video sequences by a subset of frames can maintain sign intelligibility.
- To compare the effectiveness of activity-index-based frame selection with regular frame selection.
Main Methods:
- ASL video sequences were analyzed in both dynamic and static modes.
- An activity index was developed to identify critical frames at event boundaries.
- Sign intelligibility was assessed by 32 experienced ASL signers viewing subsampled sequences.
Main Results:
- Activity-index subsampling significantly improved sign intelligibility in dynamic, full gray-scale ASL videos compared to regular frame selection.
- This improvement was even more pronounced for static images.
- The advantage of activity subsampling was less significant for binary images.
Conclusions:
- Event boundaries can be computationally defined.
- Subsampling frames from event boundaries is a superior method for preserving sign intelligibility compared to regular interval selection.