The paper introduces Discriminative World Models for Web Agents, proposing a predicted-state matching objective that trains world models to produce representations that can distinguish the true resulting state from those of alternative actions. Using a branching dataset from WebArena Go-Browse, the authors demonstrate that this approach outperforms traditional supervised next-state prediction on a held‑out benchmark and improves action ranking on WebPRMBench. Additionally, employing the discriminative world model for test‑time action selection boosts end‑to‑end task success on WebArena‑Lite.
Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PR...
The paper introduces a new training paradigm for text-based world models that prioritizes behavior consistency over traditional state consistency metrics. It proposes the Behavior Consistency Reward (BehR), a step-level metric that evaluates how the likelihood of a logged next action changes between real and predicted states using a frozen Reference Agent. Experiments on WebShop and TextWorld demonstrate that BehR-based training improves long-term alignment, reduces false positives in offline evaluation, and yields modest gains in lookahead planning while maintaining or enhancing single-step prediction quality.
arXiv:2608. 00335v1 Announce Type: cross Abstract: Compact web agents can reduce deployment cost, but training them poses challenges in both data collection and post-SFT reinforcement learning (RL).
arXiv:2606. 09961v1 Announce Type: cross Abstract: Training large language models (LLMs) as autonomous agents via reinforcement learning (RL) has enabled frontier models to achieve superhuman performance in long-horizon tasks.
arXiv:2607. 27973v1 Announce Type: new Abstract: Recently, Reinforcement Learning (RL) has emerged as a crucial paradigm for the post-training of Large Language Model (LLM) agents.
CoMAP introduces a framework that jointly evolves textual world models and agent policies through a closed‑loop interaction. At each decision step the world model forecasts future state feedback for candidate actions, while the agent reflects on the reliability of this feedback to refine its action. The resulting on‑policy trajectories are used to self‑distill and update the world model, improving prediction accuracy and long‑horizon decision‑making across embodied planning, web navigation, and tool‑use benchmarks.
arXiv:2606. 02372v1 Announce Type: new Abstract: Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution.
arXiv:2501. 14622v5 Announce Type: replace Abstract: Learning efficient representations for decision-making policies is a challenge in imitation learning (IL).
arXiv:2607. 23263v1 Announce Type: new Abstract: Deciding whether a trajectory actually fulfills its instruction governs how we measure computer-use agents on long-horizon graphical-user-interface tasks and how we train them with reinforcement learning.
arXiv:2608. 10232v1 Announce Type: cross Abstract: Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation.
arXiv:2607. 13988v1 Announce Type: new Abstract: Multi-turn agents solve complex tasks through extended sequences of tool interactions before producing a final answer, making credit assignment a fundamental challenge during post-training.
The paper introduces QWM, a framework that integrates world models with standard Q‑learning to perform test‑time search over imagined trajectories. By training the policy and value function solely on real transitions, QWM avoids compounding model bias while still benefiting from predictive search. Experiments on the Robomimic and LIBERO manipulation benchmarks show that QWM outperforms strong prior state‑of‑the‑art methods in both sample efficiency and performance.