Video Classification of Cloth Simulations: Deep Learning and Position-Based Dynamics for Stiffness Prediction

Area of Science:

• Computer Graphics
• Machine Learning
• Physics Simulation

Background:

• Virtual reality (VR), augmented reality (AR), and animation require accurate simulation of real-world deformable object movement.
• Representing cloth dynamics in virtual environments necessitates precise material properties, like stiffness.
• Current methods for extracting material properties from video for simulation are limited.

Purpose of the Study:

• To develop a deep learning (DL) method for automatically extracting cloth stiffness values from video scenes.
• To apply extracted stiffness values as material properties for virtual cloth simulation.
• To improve the realism of deformable object representation in virtual environments.

Main Methods:

• Utilized Transformer models combined with pre-trained architectures (DenseNet121, ResNet50, VGG16, VGG19) for video classification.
• Developed a model to characterize virtual cloth based on softness-to-stiffness labels.
• Trained and evaluated the model on a dataset of 3840 videos from a stiffness-oriented cloth simulation.

Main Results:

• Achieved an average accuracy of 99.50% in categorizing videos based on cloth stiffness.
• Demonstrated superior performance compared to alternative models like RNN, GRU, LSTM, and Transformer.
• Successfully applied extracted stiffness values for virtual cloth simulation.

Conclusions:

• The proposed DL method effectively extracts cloth stiffness from video, enabling more accurate virtual cloth simulation.
• This approach significantly enhances the realism of deformable objects in VR, AR, and animation.
• The model offers a robust and highly accurate solution for material property extraction in computer graphics.