arXiv Computer Vision

Do Image Editors Follow Depth-Dependent Blur and Aperture Response? A Rendered-Ground-Truth Pilot Audit

arXiv Computer Vision
Oct 8

Do Vision Models Learn Physical Constraints or Rendering Shortcuts? A Counterfactual Benchmark for Grounded Physical Consistency

The paper introduces a counterfactual benchmark to evaluate whether vision models understand physical constraints or merely learn rendering shortcuts. Using Mitsuba 3, it generates 5,500 synthetic images across 500 scene families, each with a clean image and ten violations (shadows, reflections, support, surface response, occlusion). Diagnostic baselines (LLaVA‑1.5‑7B, Qwen2.5‑VL‑7B, InternVL3.5‑8B) achieve 64.0–67.8 % macro‑F1 on a held‑out synthetic test set, but performance drops to 40.8 % for LLaVA‑1.5‑7B under an intervention shift, indicating reliance on rendering cues rather than true physical reasoning.

By M. Moein Esfahani, Sepehr Salem, Mohammed Alser, Vince Calhoun
arXiv Machine Learning
Aug 26

The Blending Ratio Is Not Where the Performance Is: Diagnosing Prototype Blending for Few-Shot Adaptation of Vision-Language Models

The paper investigates the blending ratio used in few‑shot adaptation of vision‑language models, which combines a zero‑shot text prototype with the mean of labeled image features. It shows that the theoretically optimal ratio—derived from a closed‑form mean‑squared error minimizer—does not align with the ratio that actually maximizes performance, falling short by an average of 8.5 points. Moreover, a leave‑one‑out estimate on the support set achieves near‑oracle performance, and validation‑free linear probes outperform even oracle‑tuned blends, indicating that the hyperparameter can be set near‑optimally without external validation data.

By Liangzhi Li, Bowen Wang, Yiming Qian, Thorsten Neumann, Xia Xie, Guangshun Li
arXiv Computer Vision
Oct 5

FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering

FactorSplat introduces a per-scene N‑dimensional Gaussian splatting proxy that allows region‑specific intensity‑to‑RGBA curves to be applied at inference, enabling appearance control in medical volume rendering. The method uses a shared functional encoder and low‑rank per‑Gaussian factors to learn residual appearance while keeping geometry and directional appearance shared across presets. Experiments on seven CT and MR scans show that FactorSplat outperforms region‑aware VEG in mean PSNR and changed‑region error across validation, interpolation, unseen composition, and out‑of‑distribution edits, achieving 1.10 to 1.52 dB PSNR gains and supporting fast TF switches at 524 FPS.

By Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu
arXiv Computer Vision
Aug 25

VERDICT: Agreement Beats Pixel-Space Verification in Real-Document OCSR

The paper introduces VERDICT, a method for validating optical chemical structure recognition (OCSR) outputs by leveraging agreement among multiple recognizers rather than pixel‑space re‑rendering. On 263 ACS journal images, agreement achieved an AUROC of 0.916, far surpassing the 0.547 AUROC of re‑rendering similarity. VERDICT was applied to PMC Open Access, yielding over 6,000 high‑precision structure labels, and is integrated into SES AI’s Molecular Universe platform for image‑based molecular search.

By Yani Guan, Dengpan Dong, Shuang Luo, Zi Wei, Joah Han, Dan Hannah, Yumin Zhang, Qichao Hu, Kang Xu
arXiv Computation and Language
Oct 22

What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion

The paper investigates how mainstream document converters lose formatting cues—specifically highlights and colours—when turning Word or PDF files into tokens for enterprise agents. Using a dataset of 76 policy documents, the authors show that this loss dramatically degrades agents’ ability to recover author‑specified requirements, dropping micro‑F1 scores from 0.690 to as low as 0.075. The study also demonstrates that explicitly adding cues back into the text can partially recover performance and that the issue is measurable and inexpensive to fix.

By Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra
arXiv AI
Aug 26

A survey detection channel overrides the pixels in an astronomical foundation model, and biases tomographic mean redshifts

The study audits the AION-1 foundation model, a 39‑modality transformer trained on over 200 million astronomical objects, and finds that its reliance on a survey detection channel—specifically the segmentation map—introduces a severe systematic bias. By keeping image tokens unchanged and editing only the segmentation map, all model outputs (flux, size, ellipticity, redshift) shift by factors of 110–4400 compared to a placebo, revealing that the model’s predictions are driven more by detection gating than by the actual light distribution. This bias propagates into cosmological analyses, shifting tomographic mean redshifts by a median 0.71 × the LSST DESC requirement and exceeding it in multiple assignments, while removing the detection channel eliminates the effect without measurable cost. whyItMatters":"The bias in the detection channel directly inflates errors in key astronomical measurements, potentially compromising the precision of cosmological studies that rely on accurate redshift estimates."

By Ihor Kendiukhov