Spatio-temporal categorization for first-person-view videos using a convolutional variational autoencoder and...

Area of Science:

• Artificial Intelligence
• Robotics
• Machine Learning

Background:

• Human ability to segment continuous information is crucial for understanding environments.
• Unsupervised segmentation is vital for robots to acquire spatial knowledge.
• Existing methods lack robustness in complex, dynamic environments.

Purpose of the Study:

• To propose HcVGH, a novel method for unsupervised spatio-temporal category learning.
• To enable mobile robots to segment first-person-view (FPV) videos for spatial understanding.
• To improve robot's ability to learn from visual input.

Main Methods:

• HcVGH combines a convolutional variational autoencoder (cVAE) with the HVGH method.
• HVGH incorporates deep generative and statistical models: hierarchical Dirichlet process, variational autoencoder, Gaussian process, and hidden semi-Markov model.
• The model is trained and evaluated using FPV videos from a simulated maze environment.

Main Results:

• HcVGH achieved an average segmentation F-measure of 0.77, outperforming baseline methods.
• The proposed method demonstrated superior performance in segmenting FPV videos.
• Experimental results indicate successful learning of maze environment movability parameters.

Conclusions:

• HcVGH effectively learns spatio-temporal categories from FPV videos, enabling robots to build spatial knowledge.
• The model's performance surpasses existing methods, suggesting its potential for real-world robotic applications.
• The ability to learn environmental parameters enhances robot adaptability and navigation capabilities.