arXiv Computer Vision
Beyond End-Task Success: How to Audit Visual Experience Retrieval in Robotics
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computation and Language
Sep 4
Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs
The study investigates how long‑video language models decide which frames to keep, compress, and reuse, testing each decision in isolation across six selection rules, three benchmarks, and two answering models. It finds that selecting frames based on queries yields the biggest performance boost, that halving spatial resolution costs little, and that reallocating saved tokens to more compressed frames can further improve accuracy. The work also highlights the importance of a unified evaluation harness to avoid misleading comparisons.
arXiv AI
5d ago
VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision
VisionQ is a new benchmark for qualitative analysis in computer vision that evaluates vision‑language models (VLMs) on criterion‑conditioned visual discrimination. It is built from over 1,800 peer‑reviewed comparison figures in CVPR and ICCV papers, linking each image crop to author‑stated visual claims through 3,911 hand‑annotated data points. The benchmark includes a 51‑leaf taxonomy of visual criteria, a protocol that hides method identities and reports accuracy per criterion, and a DPO‑tuned Gemma‑4‑E4B judge that improves accuracy on a held‑out test set.
arXiv AI
Jul 31
Visual Credit Audit for Multimodal Spatial Reasoning
arXiv:2607. 27069v2 Announce Type: cross Abstract: Closed yes/no spatial benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts.
arXiv Computation and Language
Sep 1
Physics-R1: An Audited Olympiad Corpus and Released Verifiers for Visual Physics Reasoning
arXiv:2605.14040v2 Announce Type: replace Abstract: Trackable improvement in multimodal physics reasoning rests on a training-and-evaluation system that is itself rarely verified: the corpora a model...
arXiv AI
Jun 9
How Many Tools Should an LLM Agent See? A Chance-Corrected Answer
arXiv:2605. 24660v2 Announce Type: replace-cross Abstract: Before an LLM agent can use a tool, a retrieval system must decide which candidate tools to show to the agent.
arXiv Machine Learning
Aug 28
Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation
The paper investigates the trade‑off of using a shared search‑and‑recommendation index that scores new items purely from features, thereby keeping the index open to unseen items. Experiments on public logs show that a feature‑based tower can match warm‑item performance (Recall@20 0.9595 vs 0.9510) and a lexical baseline, while a full‑catalog check is inconclusive. The study also quantifies the cost of this openness on recommendation quality across several baselines, revealing that exact full‑softmax training improves recall but is impractical at catalog scale.