MotionDreamer: Generating Continuous Intermediate Motions for Complex Motion Interpolation

Interpolating complex motions is challenging due to their nonlinear, non-rigid nature and ambiguities. Recent diffusion-based methods mitigate this challenge as a conditional generation task, optionally with linear motion control. However, they cannot handle these complex motions because their reconstruction training objective biases models toward per-frame appearance fidelity, which can lead to unnatural movement. To address this, we propose MotionDreamer, a novel framework that instills effective motion priors for all intermediate frames to video generators by regularizing diffusion training with ground truth continuous intermediate motions. The key idea is that the continuous intermediate motion can decompose large, complex, and ambiguous motions between specific intermediate and input frames into small, easy-to-estimate components, and thus can serve as a motion prior to represent motions across frames. MotionDreamer then recursively fuses estimated continuous motions for interpolation flows, which only updates a flow offset to the previous estimation. Thus, MotionDreamer interpolates complex motions with superior visual quality, being selected as the best-performing method in 75.1% of the human-study trials. To maximize the performance of MotionDreamer, we collect the first open-source large-scale dataset, VFIVid, for complex motion interpolation. The VFIVid contains over one million training and 130 evaluation high-quality clips with diverse objects and motions, such as dancing, skating, and camera movements. We will release all code and the dataset upon publication.