PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation

Chen Wang1*     Chuhao Chen1*     Yiming Huang 1     Zhiyang Dou1     Yuan Liu2     Jiatao Gu1     Lingjie Liu1    
1Univeristy of Pennsylvania      2HKUST
(*: equal contribution)

Arxiv, 2025

Page still under construction...

Our model achieves controllable and physics-grounded video generation from an initial force.

Abstract

Existing video generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability. To overcome these limitations, we introduce PhysCtrl, a novel framework for physics-grounded image-to-video generation with physical parameters and force control. At its core is a generative physics network that learns the distribution of physical dynamics across four materials (elastic, sand, plasticine, and rigid) via a diffusion model conditioned on physics parameters and applied forces. We represent physical dynamics as 3D point trajectories and train on a large-scale synthetic dataset of 550K animations generated by physics simulators. We enhance the diffusion model with a novel spatiotemporal attention block that emulates particle interactions and incorporates physics-based constraints during training to enforce physical plausibility. Experiments show that PhysCtrl generates realistic, physics-grounded motion trajectories which, when used to drive image-to-video models, yield high-fidelity, controllable videos that outperform existing methods in both visual quality and physical plausibility.

Teaser image

Given a single image, we lift the object in that image into 3D points. We train a diffusion-based trajectory generation model conditioned on physics parameters and external force for motion generation, which are then used as strong physics-grounded guidance for image-to-video generation.

BibTeX


@inproceedings{physctrl2025,
  Author = {Chen Wang* and Chuhao Chen* and Yiming Huang and Zhiyang Dou and Yuan Liu and Jiatao Gu and Lingjie Liu},
  Title = {PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation},
  Year = {2025},
  booktitle={Arxiv},
}