Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Computation and Language
Sep 30

Memory Consolidation Flattens the Temporal Shape of User Facts

The paper introduces the concept of aspectual flattening, where long‑term memory systems convert conversational statements into concise notes that omit temporal cues. Using the LAPSE benchmark, the authors show that several memory writer models consistently flatten progressive statements while preserving simple‑present ones, and this asymmetry persists across multiple configurations. Experiments reveal that the loss of temporal information can alter how downstream readers assess the validity of a fact and influence their decision‑making, sometimes leading them to act without further verification.

By Sugam Panthi, Muhaiminul Yeamin, Siyan Luo, Rabab Abdelfattah
arXiv Computer Vision
Sep 30

Beacon: Knowing When and How to Perform Agentic Visual Reasoning

Beacon is a new agentic visual reasoning model that improves multimodal large language models (MLLMs) by better deciding when to use tools and how to use them. It introduces two key concepts—Mode Adaptiveness, which ensures tools are invoked only when necessary, and Tool Effect, which measures the net benefit of tool use— and trains the model with supervised fine‑tuning and reinforcement learning that rewards necessity-aware decisions and expands capability through expert hints. Across 13 benchmarks, Beacon outperforms other open‑source models, achieving the highest average score and the largest net tool‑gain on diagnostic tests.

By Qixun Wang, Yang Shi, Letian Cheng, Zhuoran Zhang, Yan He, Yuqi Tang, Qi Zhang, Xinlei Yu, Ruizhe Chen, Tianrun Xu, Yuanxing Zhang, Pengfei Wan, Haotian Wang, Xianghua Ying
arXiv Computer Vision
Sep 30

PARSEE-VAD: Efficient Training-Free Online Video Anomaly Detection via Proposition-Aware Reasoning and Streaming Evidence Escalation

PARSEE-VAD is a training‑free online video anomaly detection framework that separates semantic evidence acquisition from score‑state evolution. It uses Proposition‑Aware Reasoning to extract structured propositional evidence from the current causal window and selectively activates more specific queries, while Streaming Evidence Escalation maps this evidence into a compact score‑domain event state and propagates only the bounded state to maintain temporal continuity. Experiments on four benchmarks show strong performance with reduced specialist computation and sparse score‑state propagation, supporting a current‑first principle for streaming multimodal inference.

By Ji Wang, Shuangqing Zhang, Guo-Sen Xie, Fang Zhao
arXiv Computer Vision
Sep 30

PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization

PGL-3D introduces a progressive geometric learning framework for 3D visual query localization, where intermediate cuboids guide feature aggregation and refinement. The method predicts a complete cuboid for each proposal, selects reference observations via Query‑Tube‑Memory, pools query‑conditioned features, and re‑predicts refined cuboids. A training‑only objective, ST‑D9O, supervises cuboid geometry at every stage, yielding significant performance gains over prior baselines.

By Liang Peng, Shizhuo Mu, Bohan Tan, Wenyuan Wang, Chen Zhao, Xingping Dong, Heng Fan, Libo Zhang, Bo Du