CALIPER is a new benchmark that tests whether pretrained visual encoders can infer physical properties such as mass and friction from images. The test involves striking an object twice at known speeds, showing a third strike only up to contact, and asking a linear readout on frozen features to predict how far the object slides. Results show that in clean, fixed‑camera scenes all representations perform similarly, but when camera, lighting, and clutter are varied, only encoders that truly infer physics—like V‑JEPA 2—maintain performance, while random or raw pixel representations fail.
arXiv:2609.09528v1 Announce Type: new Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: h...
The paper introduces a counterfactual benchmark to evaluate whether vision models understand physical constraints or merely learn rendering shortcuts. Using Mitsuba 3, it generates 5,500 synthetic images across 500 scene families, each with a clean image and ten violations (shadows, reflections, support, surface response, occlusion). Diagnostic baselines (LLaVA‑1.5‑7B, Qwen2.5‑VL‑7B, InternVL3.5‑8B) achieve 64.0–67.8 % macro‑F1 on a held‑out synthetic test set, but performance drops to 40.8 % for LLaVA‑1.5‑7B under an intervention shift, indicating reliance on rendering cues rather than true physical reasoning.
arXiv:2610.31983v1 Announce Type: new Abstract: Video encoder models are increasingly used as world models for physical AI, where a policy plans by rolling out the latent, so what physics the latent...
STATERA is a method that adapts a pretrained video backbone with mostly frozen weights and a lightweight temporal tubelet mixer to estimate the center-of-mass (CoM) of opaque, asymmetric rigid bodies from short monocular videos. It introduces the HiddenMass Benchmark, consisting of 50K simulated MuJoCo trajectories and a 63-sequence real-world test set with calibrated CoM ground truth. In simulation, STATERA reduces normalized CoM error from 41.7% to 25.2%, and in zero-shot sim-to-real transfer, its phase‑aware variant consistently predicts movement toward the true hidden offset, improving physics capture from 2.6% to 41.0%.
The paper introduces Rate‑Controlled Refresh (RCR), a method for re‑selecting key‑value (KV) cache entries during long‑form multimodal generation. RCR lets attention drift determine when and where to refresh, while a per‑sample controller limits how often this occurs, and a low‑rank drift detector keeps the process GPU‑resident. Across multiple models and tasks, RCR outperforms the per‑step retrieval method InfiniGen by up to 3.8 ROUGE‑L points and remains close to exact per‑step re‑selection, with reduced data movement and a smaller GPU‑resident detector.
arXiv:2610.07355v1 Announce Type: cross Abstract: Video world models track objects they can see; we ask what they keep of objects they cannot. We hide an object from a frozen V-JEPA 2 predictor and c...
arXiv:2609.13235v1 Announce Type: cross Abstract: Networked manipulation endpoints couple perception to actuation across compute- and bandwidth-limited links, yet commonly exchange dense geometric st...
arXiv:2608. 15555v1 Announce Type: cross Abstract: Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whether the predicted objects move correctly.
arXiv:2609.13308v1 Announce Type: cross Abstract: A companion evaluation found that naming the target part in a manipulation prompt increased action accuracy by 0.32-0.63 across eight vision-language...
arXiv:2607. 13305v1 Announce Type: cross Abstract: Benchmark accuracy in video large language models (LLMs) is often treated as evidence of visual understanding.
A robot must understand the state of its own body, but a camera sees only part of it. Force and contact leave almost no trace in a single frame, and raw vision features read force at $R^2$ at or below $0.
The paper introduces a new benchmark for vision‑language models that tests their ability to decide whether to answer a physics question immediately or to request additional experimental evidence. Each problem presents one measurement image and four possible physical worlds defined by two masses and two values of another property; the model must either stop and answer or choose the cheapest experiment that resolves the question. Across six open models and 144 parameter sets, the models almost always repeat the same action even when the optimal choice changes, and only a single model gets both decisions correct on 5.9% of cases.