← Back to all news
arXiv Machine Learning

Beyond State Consistency: Behavior Consistency in Text-Based World Models

The paper introduces a new training paradigm for text-based world models that prioritizes behavior consistency over traditional state consistency metrics. It proposes the Behavior Consistency Reward (BehR), a step-level metric that evaluates how the likelihood of a logged next action changes between real and predicted states using a frozen Reference Agent. Experiments on WebShop and TextWorld demonstrate that BehR-based training improves long-term alignment, reduces false positives in offline evaluation, and yields modest gains in lookahead planning while maintaining or enhancing single-step prediction quality.

Related stories

arXiv AI
Sep 4

CoMAP: Co-Evolving World Models and Agent Policies for LLM Agents

CoMAP introduces a framework that jointly evolves textual world models and agent policies through a closed‑loop interaction. At each decision step the world model forecasts future state feedback for candidate actions, while the agent reflects on the reliability of this feedback to refine its action. The resulting on‑policy trajectories are used to self‑distill and update the world model, improving prediction accuracy and long‑horizon decision‑making across embodied planning, web navigation, and tool‑use benchmarks.

By Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li
llmsagentsroboticsefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Sep 2

Discriminative World Models for Web Agents

Discriminative World Models for Web Agents proposes a new training objective called predicted‑state matching, which forces a world model to produce representations that can distinguish the true resulting web state from those produced by alternative actions. The authors train these models on a branching dataset from WebArena Go‑Browse, where each decision point includes multiple actions and their outcomes. Experiments show that models trained with predicted‑state matching outperform those trained with standard supervised next‑state prediction on a held‑out benchmark, improve PRM‑style action ranking on WebPRMBench, and enhance end‑to‑end task success on WebArena‑Lite when used for test‑time action selection.

agentsreinforcement-learningbenchmarks
More like this →
arXiv AI
Sep 3

Discriminative World Models for Web Agents

The paper introduces Discriminative World Models for Web Agents, proposing a predicted-state matching objective that trains world models to produce representations that can distinguish the true resulting state from those of alternative actions. Using a branching dataset from WebArena Go-Browse, the authors demonstrate that this approach outperforms traditional supervised next-state prediction on a held‑out benchmark and improves action ranking on WebPRMBench. Additionally, employing the discriminative world model for test‑time action selection boosts end‑to‑end task success on WebArena‑Lite.

By Kelvin Li, Dhruv Pendharkar, Anish Pahilajani, Chuyi Shang, Leon Oks, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, Roei Herzig
agentsreinforcement-learningbenchmarks
More like this →
arXiv AI
Oct 1

Beyond Prediction: Steering VLM Agents with Retrospective World Modeling

The paper introduces Retrospective World Modeling, a new paradigm for vision‑language‑model (VLM) agents that allows them to reason backward by estimating which action most likely caused a state transition. It proposes the Self‑Consistency Reward (SCR), an intrinsic signal that measures how well a policy action aligns with this retrospective explanation, providing dense transition‑level feedback. Experiments demonstrate that incorporating SCR improves policy robustness and generalization compared to purely prospective world‑modeling approaches.

By Yongjiang Liu, Jie Zhang, Haoyue Zhang, Jingcai Guo, Deze Zeng, Song Guo
agentsreinforcement-learningmultimodal
More like this →