OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
arXiv:2607. 28609v2 Announce Type: replace Abstract: Computer-using agents (CUAs) are advancing rapidly across the digital world.
The paper introduces SGV, a self‑grounded verification method that mitigates agreement bias in multimodal large language models (MLLMs). By first generating broad priors about desired behavior and then conditioning on those priors to evaluate agent trajectories, SGV improves human alignment and detection of failures across web navigation, computer use, and robotics tasks. Experiments on 13 models and 28 judge designs show up to 25 percentage‑point gains in failure detection and significant performance boosts in downstream applications such as GUI task completion and robotic policy learning.
arXiv:2607. 28609v2 Announce Type: replace Abstract: Computer-using agents (CUAs) are advancing rapidly across the digital world.
arXiv:2512. 03438v3 Announce Type: replace Abstract: Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final answers.
arXiv:2606. 31270v1 Announce Type: cross Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility.
arXiv:2607.28225v2 Announce Type: replace Abstract: Agentic vision-language models (VLMs), which interleave textual reasoning with explicit tool calls such as cropping and code-based image manipulati...
Beacon is a new agentic visual reasoning model that improves multimodal large language models (MLLMs) by better deciding when to use tools and how to use them. It introduces two key concepts—Mode Adaptiveness, which ensures tools are invoked only when necessary, and Tool Effect, which measures the net benefit of tool use— and trains the model with supervised fine‑tuning and reinforcement learning that rewards necessity-aware decisions and expands capability through expert hints. Across 13 benchmarks, Beacon outperforms other open‑source models, achieving the highest average score and the largest net tool‑gain on diagnostic tests.
The paper introduces MOTIVE, a multi-view self-verification framework for vision-language models that evaluates candidate answers from complementary perspectives and learns a reliability score aligned with correctness. By using this score to decide whether to accept an answer or trigger history-guided rethinking, MOTIVE improves reliability and efficiency compared to existing self-verification and self-correction methods. Extensive experiments on diverse multimodal benchmarks and VLM backbones show consistent performance gains and reduced unnecessary reasoning turns.
AgentHorizon is a benchmark comprising 1,373 computer‑use tasks with instruction‑trajectory pairs collected from 166 hours of human‑recorded activity across three operating systems. The benchmark uses a paired design that swaps instructions to create negative tasks, testing judges’ ability to distinguish successful trajectories from those that completed a similar but incompatible request. Eleven judges were evaluated, with GPT‑5.5 achieving the highest balanced accuracy of 80.9% on the full trajectory subset, while tool‑use benefits some models but harms others, revealing large variability in judges’ performance.
MineExplorer is a benchmark designed to assess the open‑world exploration abilities of multimodal large language models (MLLMs) in Minecraft. It filters out tasks that rely heavily on Minecraft‑specific knowledge, organizes tasks into ReAct‑style capabilities, and composes atomic tasks into implicit multi‑hop challenges. A multi‑agent synthesis workflow creates reliable task graphs, sandbox scenes, and rule‑based milestone evaluators, and human evaluation confirms its superiority over a single‑agent baseline. Experiments show that while advanced MLLMs can handle many single‑hop tasks, they struggle with longer trajectories that require coordinating hidden prerequisites, and larger models or different thinking modes do not consistently improve performance.
MLExpert is a 35B Mixture-of-Experts agentic model designed to autonomously perform long‑horizon machine learning engineering (MLE) tasks. It uses a post‑training method that combines hierarchical agentic supervised fine‑tuning with Windowed Interaction RL to learn from intermediate outcomes in complex experimentation workflows. On the MLE‑Bench Lite benchmark, MLExpert raises the average medal rate from 37.9% to 72.7%, matching the performance of larger frontier models and demonstrating generalizable optimization capabilities across related tasks.
arXiv:2601. 02854v2 Announce Type: replace Abstract: As an agent-level reasoning and coordination paradigm, Multi-Agent Debate (MAD) orchestrates multiple agents through structured debate to improve answer quality and support complex reasoning.
SWE-VERIFIER is a lightweight generative model that selectively triggers candidate sampling and verification for coding agents, scoring proposed actions to decide when additional sampling is needed and ranking the resulting actions before execution. Trained on Qwen3.5-4B with preference data, it improves resolution on MiniMax-M2.7 from 75.47% to 77.20% at 1.09× cost and on DeepSeek-V4-Flash from 72.60% to 74.27% at 1.13× cost, while verifying fewer than 4% of steps. The approach demonstrates that selective sampling and lightweight verification can enhance coding‑agent performance and inference efficiency with minimal overhead.
arXiv:2605. 14054v2 Announce Type: replace Abstract: Achieving robust perception-reasoning synergy is a central goal for advanced Vision-Language Models (VLMs).