Visualizing the Passage of Time with Video Temporal Pyramids

Area of Science:

• Computer Vision
• Data Visualization
• Time Series Analysis

Background:

• Analyzing long-term video data presents challenges due to sheer volume and slow-moving phenomena.
• Traditional timelapses suffer from aliasing and fixed temporal resolution, limiting their effectiveness.

Purpose of the Study:

• To introduce Video Temporal Pyramids (VTP) for effective visualization of multi-timescale phenomena in long videos.
• To develop a complementary Video Spectrogram for exploring scene dynamics across different temporal resolutions.

Main Methods:

• Developed a temporal pyramid algorithm, analogous to spatial image pyramids, to represent video data at multiple timescales.
• Constructed Video Temporal Pyramids from months or years of outdoor scene data.
• Introduced a Video Spectrogram to visualize activity across pyramid levels.

Main Results:

• VTP layers enable alias-free viewing of long-term changes, outperforming naive timelapses.
• The Video Spectrogram effectively aids in exploring and discovering phenomena across various timescales.
• Demonstrated the technique on ten diverse outdoor scenes with extensive data.

Conclusions:

• Video Temporal Pyramids provide a powerful tool for understanding temporal dynamics in long videos.
• The Video Spectrogram enhances exploration and discovery of events across different time scales.
• This approach expands possibilities for visualizing and analyzing time-dependent phenomena.