The paper introduces DualViewEval, a benchmark compression technique for agent evaluation that jointly uses outcome and process signals to learn an exact-size miniset and predict full-benchmark scores. By analyzing large-scale trajectories, the authors identify six complementary process signals linked to final agent performance. Across five agent benchmarks and five baselines, DualViewEval achieves superior compression (24×–40×) and lower error rates, while also revealing capability differences among agents for efficient development.
arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.
arXiv:2608. 16798v1 Announce Type: cross Abstract: Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment.
arXiv:2605. 26494v2 Announce Type: replace-cross Abstract: We introduce the MiniMax-M2 series, a family of Mixture-of-Experts language models built around the principle that mini activations can unleash maximum real-world intelligence.
The paper investigates two strategies for improving large language model (LLM) evaluation: specialized judge weights and rule‑based deferral policies. Experiments on nearly 100,000 rubric‑conditioned samples show that correct rubrics boost accuracy, while incorrect ones hurt it, and that splitting training data into criterion‑specific experts can severely degrade performance unless the experts are warm‑started from a unified model. The authors demonstrate that lightweight deferral cascades can match or exceed the accuracy of larger standalone judges at a fraction of the compute cost, and they provide practical design rules for building efficient, reliable LLM evaluators.
arXiv:2607. 02032v1 Announce Type: new Abstract: Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure.
arXiv:2607. 09375v1 Announce Type: new Abstract: We present Mach-Mind-4-Flash, a 35B-parameter Mixture-of-Experts (MoE) agentic model with 3B activated parameters.
arXiv:2605. 28556v2 Announce Type: replace Abstract: As agent capabilities advance, existing benchmarks, such as $\tau^2$-Bench, are becoming increasingly saturated.
arXiv:2608. 11434v1 Announce Type: new Abstract: Mobile agent benchmarks increasingly rely on LLM-based judges to evaluate task completion, yet the reliability of these judges on mobile agent trajectories remains largely unexamined.
arXiv:2608.29168v1 Announce Type: new Abstract: The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model...
EarlyEval introduces a lightweight framework that predicts an LLM agent’s final outcome early in its execution, allowing the run to halt when a LightGBM classifier reaches a calibrated confidence threshold. By training success and failure classifiers on behavioral, textual, and reference-solution features, EarlyEval can cut 13%-26% of agent steps and up to 44.1% of input tokens while maintaining 89%-97% prediction accuracy. Across three benchmarks—SWE-bench Verified, TerminalBench, and Toolathlon—this approach reduces evaluation costs with minimal impact on per-agent resolve rates.
EviRover is a perception agent that goes beyond a single glance by actively gathering information to resolve perceptual queries. The authors created two data generation pipelines, producing EviRover-SFT-5K and EviRover-RL-12K, and a human‑verified benchmark called EviLens with 688 instances across five perception categories. Trained with supervised fine‑tuning and agentic reinforcement learning, the 4B EviRover outperforms its backbone by an average of 30 points on EviLens and shows strong transfer to other benchmarks such as WebEyes and BrowseComp‑VL.
arXiv:2607. 29626v1 Announce Type: new Abstract: As LLMs evolve from code completion systems into autonomous scientific agents, evaluating their ability to conduct experiments has become increasingly important.