Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv AI
Oct 2

ReForge: Refining Merged Models with Anchor-Regularized Regression

ReForge is a bilevel optimization framework that refines merged models by treating module-wise refinement as Bayesian linear regression with an anchor-centered prior. The inner level produces a closed‑form MAP estimate from unlabeled calibration activations, while the outer level employs Bayesian optimization to jointly select regularization strengths and assembly scales using validation data. A data‑free variant replaces activation statistics with task‑vector Grams, enabling refinement without calibration examples, and across extensive vision and language benchmarks ReForge consistently outperforms existing plug‑and‑play anchor baselines, achieving significant accuracy gains on large‑scale tasks such as 20‑task ViT‑B/32 and eight‑task ViT‑L/14.

By Kaiyang Li, Shaobo Han, Qing Su, Shihao Ji
arXiv AI
Oct 2

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

Selection-Based Structured Reasoning (SSR) is a framework that replaces free-form reasoning with the selection of pre-specified natural‑language reasoning candidates, allowing multimodal agents to choose from a set of reusable high‑level reasoning traces. By scoring these candidates in parallel using a shared context KV cache, SSR eliminates the need for an auxiliary task head and reduces inference cost. Experiments on seven multimodal search benchmarks with 2B and 4B models show that SSR maintains competitive success rates while cutting per‑turn reasoning latency by over 90% and overall inference latency by 28–54%.

By Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang, Rui Yang, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh, Reid Simmons, Joerg Liebelt, Xiang Kong, Zhongyu Jiang
arXiv Machine Learning
Oct 2

OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning

arXiv:2609.40265v1 Announce Type: new Abstract: Real-world time-series applications increasingly require models that can handle time series forecasting, context-conditioned prediction, and language-b...

By Tony Chen, Timo Stoffregen, Maxwell Xu, Thomas Kaar, Martin Maritsch, Geremia Pompei, Nicolas Zumarraga, Robert Jakob, Paul Schmiedmayer, Patrick Langer, Juncheng Liu
arXiv Machine Learning
Oct 2

STARS: From Spatiotemporal Dynamics to Social Representations in Human-Robot Interaction

arXiv:2609.40245v2 Announce Type: cross Abstract: Robot navigation in dynamic, human-centered environments requires socially-compliant decisions grounded in robust scene understanding. Recent Vision-...

By Nathan Tsoi, Michael J. Munje, Tejas Oberoi, Rishab Maheshwari, Pengen Zheng, Tanush Chauhan, Peter Stone, Joydeep Biswas
arXiv AI
Oct 2

PickMoment: Continuous-Time Single-Image-to-Video via Learning Deblurring and Blur-to-Video

PickMoment is a continuous‑time model that learns to predict the interval‑mean blur over arbitrary sub‑intervals of a camera exposure, unifying single‑image deblurring, blur‑to‑video generation, and continuous‑time pick‑a‑moment recovery. It is trained with three supervisions derived from the blur integral: an empirical reconstruction loss, an additivity loss for self‑consistency, and a sharp‑frame loss at zero interval. The model achieves state‑of‑the‑art performance on GoPro and HIDE for generative deblurring, competitive results on RealBlur, and the highest per‑frame fidelity on GoPro‑7 blur‑to‑video, all in a single forward pass.

By Junseong Shin, Hyeonsu Jo, Daehyun Kim, Tae Hyun Kim
arXiv AI
Oct 2

HHR: Hierarchical Hash Retrieval for Efficient LLM Generation

The paper introduces Hierarchical Hash Retrieval (HHR), a coarse‑to‑fine framework designed to improve hash‑based retrieval for large language models. HHR combines Geometry‑Aware Key Routing (GKR) to redistribute feature magnitudes and prune low‑logit keys, with Learned Hash Projection (LHP) to align Hamming distance with true query‑key relevance for fine‑grained retrieval. Experiments on diverse LLMs and benchmarks show that HHR outperforms existing methods, boosting LongBench scores by 1.10 points and achieving up to 3.30× decoding speedup at 128K context length for Llama‑3.1‑8B‑Instruct.

By Lianjun Liu, Tiantian Zheng, You Huang, Weiqi Yan, Mingte Qiu, Huazhong Liu, Xiaofeng Zhu, Yunshan Zhong
arXiv AI
Oct 2

MoLE: Mixture of Latent Experts for Complementary Visual Reasoning

MoLE introduces a Mixture of Latent Experts framework that encourages distinct latent tokens to extract complementary visual information, rather than sharing visual evidence. By isolating latent visual experts during evidence extraction and aggregating their outputs with dedicated summary experts, MoLE achieves higher performance on visual reasoning benchmarks compared to existing latent reasoning methods. The approach uses a two-stage training pipeline that first forces visual evidence through the latent pathway and then restores direct visual access, without requiring predefined expert roles or intermediate visual targets.

By Yingcheng Liu, Tianyi Jiang, Yujuan Ding, jiangbo Ai, Xun Jiang, Guoqing Wang, Wei Ye, Yi Bin