arXiv Computer Vision
Interpreting Physics in Video Encoders using Multi-Object Interaction Videos
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 4
Principia: Relational Physics Tests for Video Models
Principia is a new benchmark that tests video models on Newtonian physics by evaluating relational consistency between paired objects, independent of calibration. It covers eight phenomena—gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass‑spring oscillation—across translational, rotational, collisional, and oscillatory dynamics using real‑world scenes. The benchmark introduces a calibration‑independent consistency score and shows that current state‑of‑the‑art video generators perform poorly on it, with the best vision‑language model achieving only 67% accuracy on detecting physics violations.
arXiv AI
Sep 10
CALIPER: Clean Scenes Cannot Rank Physical Inference in Pretrained Visual Representations
CALIPER is a new benchmark that tests whether pretrained visual encoders can infer physical properties such as mass and friction from images. The test involves striking an object twice at known speeds, showing a third strike only up to contact, and asking a linear readout on frozen features to predict how far the object slides. Results show that in clean, fixed‑camera scenes all representations perform similarly, but when camera, lighting, and clutter are varied, only encoders that truly infer physics—like V‑JEPA 2—maintain performance, while random or raw pixel representations fail.
arXiv Computer Vision
Sep 21
CompAdapt: Adaptable Composite Motion Modeling for Physics-Consistent Text-to-Video Generation
CompAdapt is a physics-consistent text-to-video generation framework that extends diffusion-based models to handle composite physical behaviors such as coupled motions, multi-stage transitions, and multi-object collisions. It translates natural language prompts into structured physical semantics, enabling end-to-end specification of motion types, temporal relations, and initial parameters. The system introduces dynamics-aware prior matching for one-shot adaptation to new physical environments and a physics-aware latent feature fusion module to enhance visual fidelity during fast, complex motion, outperforming existing physics-constrained baselines on physics-focused T2V benchmarks.
arXiv AI
Jun 29
PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation
arXiv:2606. 28128v1 Announce Type: cross Abstract: Video generation models have emerged as a promising paradigm for embodied world simulation.
arXiv AI
Sep 15
How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms
arXiv:2606.09646v2 Announce Type: replace-cross Abstract: We study whether pretrained video foundation models encode intuitive-physics information in their frozen representations, and how this inform...
arXiv AI
Aug 7
GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models
arXiv:2608. 05948v1 Announce Type: new Abstract: Physics engines facilitate large-scale training and evaluation for embodied intelligence, while generative video world models are emerging as implicit simulators of future states and interactions.