arXiv Machine Learning

The Camera Inside the Editor: Reading the Implicit Camera of Image Editors with Painted Calibration Patterns

arXiv AI
Sep 21

PACE: Precise AI Cinematic Expression

arXiv:2609.19853v2 Announce Type: replace-cross Abstract: Between a screenplay and a film sits a planning problem that is spatial first: who stands where, and what a camera sees from where it stands....

By Bing Duan, Qiang Guo, Linpu Li, Zhijian Mao, Min Zhu, Zhirui Ren, Yiwei Yan, Xi Chu, Xiaoding Li
arXiv AI
Sep 10

CALIPER: Clean Scenes Cannot Rank Physical Inference in Pretrained Visual Representations

CALIPER is a new benchmark that tests whether pretrained visual encoders can infer physical properties such as mass and friction from images. The test involves striking an object twice at known speeds, showing a third strike only up to contact, and asking a linear readout on frozen features to predict how far the object slides. Results show that in clean, fixed‑camera scenes all representations perform similarly, but when camera, lighting, and clutter are varied, only encoders that truly infer physics—like V‑JEPA 2—maintain performance, while random or raw pixel representations fail.

By Aman Mehta, Riya Baviskar
arXiv AI
Sep 18

PACE: Precise AI Cinematic Expression: A Typed Specification for Script-Grounded Previsualization and Geometric Conformance

PACE (Precise AI Cinematic Expression) is a typed specification that captures a film’s spatial plan—screenplay evidence, characters, props, locations, and camera actions—at script, scene, shot, or panel levels, with inheritance to lower levels. A compiler transforms this plan into prompts for diffusion models and a metrically accurate 3D scene, while a camera solver ensures the declared framing is achieved. Evaluation on the Automatic Drive screenplay and 204 external shots shows high positional accuracy and improved action capture when poses are declared, though transitions and motion remain future work.

By Bing Duan, Qiang Guo, Linpu Li, Zhijian Mao, Min Zhu, Zhirui Ren, Yiwei Yan, Xi Chu, Xiaoding Li
arXiv Computer Vision
Oct 8

Do Vision Models Learn Physical Constraints or Rendering Shortcuts? A Counterfactual Benchmark for Grounded Physical Consistency

The paper introduces a counterfactual benchmark to evaluate whether vision models understand physical constraints or merely learn rendering shortcuts. Using Mitsuba 3, it generates 5,500 synthetic images across 500 scene families, each with a clean image and ten violations (shadows, reflections, support, surface response, occlusion). Diagnostic baselines (LLaVA‑1.5‑7B, Qwen2.5‑VL‑7B, InternVL3.5‑8B) achieve 64.0–67.8 % macro‑F1 on a held‑out synthetic test set, but performance drops to 40.8 % for LLaVA‑1.5‑7B under an intervention shift, indicating reliance on rendering cues rather than true physical reasoning.

By M. Moein Esfahani, Sepehr Salem, Mohammed Alser, Vince Calhoun
arXiv Computer Vision
Sep 18

Printing the Underdetermined: Materializing Multi-solutionness in Figurative Paintings

The paper challenges the common assumption that figurative paintings represent a single, recoverable 3D scene. It introduces the concept of multi-solutionness, highlighting how unobserved content and ambiguous visual cues allow multiple plausible 3D configurations. The authors present a workflow that generates multiple camera-orbit video sequences from a single painting, reconstructs each with 3D Gaussian Splatting, and fabricates the resulting point-based Gaussian scenes as physical artifacts via DreamPrinting, thereby making the non-uniqueness of interpretations explicit and inspectable.

By Yutao Ming, Teng Xu, Youjia Wang, Yunyang Liu, Fengmin Yang, Fuqiang Zhao, Jingyi Yu, Hua Yang, Yanjun Zhou
arXiv AI
Sep 24

What Looks Like a Capability Limit in Vision-Language Models Is a Readout Limit

The paper argues that apparent capability limits in vision‑language benchmarks often stem from the way answers are presented rather than from the models themselves. By comparing performance on COCO images with answer choices given as English names versus pixel coordinates, the authors show that models like Qwen3‑VL‑4B perform far better when answers are in natural language, and that the choice of answer format can swing model rankings by dozens of points. The study also demonstrates that different conventions (e.g., hue angles vs. pixel coordinates) reveal which formats a model can actually interpret, highlighting that a fixed answer vocabulary is not neutral across models.

By Alfredo F. Frontera Del Valle
arXiv Computer Vision
Oct 16

Every View Counts: View-Consistent Panoptic Quality for Multi-view Panoptic Segmentation

The paper introduces View‑Consistent Panoptic Quality (VC‑PQ), a new metric for evaluating multi‑view panoptic segmentation that treats each visible view equally and penalizes inconsistencies between predicted and ground‑truth visibility. VC‑PQ extends the traditional per‑scene PQ by decomposing the score into mask accuracy, view consistency, and matching threshold, and includes a parameter to recover area weighting for comparison. Experiments on ScanNet++ and ScanNetv2 show that VC‑PQ better reflects missed detections or ID changes across views than the conventional PQ^scene metric.

By Youngmin Lee, Byungha Ko, Guhnoo Yun, Dong Hwan Kim