Video to Video Generative Adversarial Network for Few-Shot Learning Based on Policy Gradient

Area of Science:

• Artificial Intelligence
• Computer Vision

Background:

• Deep reinforcement learning (RL) and generative adversarial networks (GANs) have advanced video-to-video synthesis.
• Existing methods often require paired video inputs, limiting their generalizability.

Purpose of the Study:

• To propose RL-V2V-GAN, a new deep neural network for unsupervised conditional video-to-video synthesis.
• To develop a method that maps source to target video domains while preserving source style.

Main Methods:

• Utilized deep reinforcement learning (RL) and generative adversarial networks (GANs).
• Employed policy gradient for training and convolutional long short-term memory (ConvLSTM) layers for spatiotemporal information.
• Designed a fine-grained GAN architecture with spatiotemporal adversarial goals.

Main Results:

• Achieved content translation while preserving video style.
• Demonstrated effectiveness in few-shot learning scenarios due to unsupervised, unpaired input approach.
• Produced temporally coherent video synthesis results.

Conclusions:

• RL-V2V-GAN offers a more general approach to video-to-video synthesis compared to paired methods.
• The model shows significant potential for advancing unsupervised conditional video synthesis.
• Effective for scenarios with limited target domain data.