arXiv AI
Aug 10
Alignment has a Fantasia Problem
arXiv:2604. 21827v2 Announce Type: replace Abstract: In accomplishing complex tasks, human cognition typically progresses from abstract to concrete (e.
Interactive Task Alignment as a POMDP
arXiv:2607. 16412v1 Announce Type: new Abstract: Current benchmarks for language models primarily evaluate execution on fully specified tasks.
Related stories
arXiv AI
3d ago
Beyond Oracle Communication: Benchmarking Interactive Intent Alignment Under Miscommunication and Evolving User Intent
arXiv:2609.38604v1 Announce Type: cross Abstract: Modern LLM agents increasingly tackle complex tasks through interactive, long-horizon exchanges with users, while existing benchmarks generally assum...
arXiv Computation and Language
Sep 1
Beyond Surface Alignment: Grounding the Dynamics of Situational Understanding and Generative Control in LLMs
arXiv:2608.29610v1 Announce Type: new Abstract: The current alignment tuning paradigm for Large Language Models (LLMs) prioritizes surface-level behaviors -- fluency, safety, and tonal consistency. W...
arXiv AI
Jul 24
Expectation Alignment of Language Models for Real-World User Expectations
arXiv:2607. 20485v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable performance on standard benchmarks, yet it remains largely unexplored whether they truly meet user expectations.
arXiv Machine Learning
Sep 3
Beyond State Consistency: Behavior Consistency in Text-Based World Models
The paper introduces a new training paradigm for text-based world models that prioritizes behavior consistency over traditional state consistency metrics. It proposes the Behavior Consistency Reward (BehR), a step-level metric that evaluates how the likelihood of a logged next action changes between real and predicted states using a frozen Reference Agent. Experiments on WebShop and TextWorld demonstrate that BehR-based training improves long-term alignment, reduces false positives in offline evaluation, and yields modest gains in lookahead planning while maintaining or enhancing single-step prediction quality.
arXiv AI
Sep 25
IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents
IDRBench is a benchmark designed to evaluate the interactive capabilities of deep research agents that use large language models. It introduces controlled opportunities for clarification within a common workflow, comparing autonomous and interactive trajectories by measuring task‑specific report alignment and interaction cost. Experiments on 100 tasks with seven LLMs show that interaction consistently improves alignment, though its effectiveness varies depending on the agents’ questions and feedback integration.
arXiv AI
Sep 3
Thinking effort aligns between humans and reasoning models in abductive reasoning
The study examines how the effort expended by large reasoning models (LRMs) compares to that of humans during abductive reasoning tasks. By analyzing reaction times and reasoning traces, the authors find that LRMs and humans exhibit similar patterns of effort and error types. They also demonstrate that decoding strategies allowing models to explore multiple reasoning paths further align the models’ reasoning costs with human effort.
arXiv AI
Aug 12
SPIEval: Evaluating Large Language Models as Mobile Assistants over Scattered Personal Information
arXiv:2608. 10692v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as mobile assistants, where a key challenge is leveraging personal information scattered across multiple applications (apps) to complete user instructions.
arXiv AI
Aug 26
MetaRAG: Belief-Action Aligned Policy Optimization for Agentic RAG
MetaRAG introduces a belief-action aligned policy optimization framework for agentic retrieval-augmented generation (RAG). It incorporates Verify-first Action Generation and Internal Belief Probing to assess whether the current evidence is sufficient before taking an action, and uses a consistency reward gated by answer correctness to guide training. Experiments on seven public QA benchmarks demonstrate that MetaRAG improves the accuracy-efficiency trade-off over existing RL-based agentic RAG baselines, with benefits that transfer across research settings, optimizers, and model backbones.
arXiv AI
Jun 26
Diagnosing Task Insensitivity in Language Agents
arXiv:2606. 26918v1 Announce Type: new Abstract: Large language models can serve as capable long-horizon agents, but their out-of-distribution (OOD) generalization remains weak.
arXiv AI
Sep 16
DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow
arXiv:2509.12626v4 Announce Type: replace-cross Abstract: Aligning agentic AI with user intent is critical for delegating complex, socially embedded tasks, yet user preferences are often implicit, ev...
arXiv AI
Aug 19
KnowSim: Evaluating Information Calibration in LLM Assistants with User Simulators that Learn
KnowSim introduces an evaluation framework that uses a user simulator with explicit knowledge states to assess how well large language models calibrate information to users. The simulator represents knowledge as a graph of Information Units with prerequisite relationships and updates these states based on learning theory. KnowSim computes Knowledge Gain, Delivery Calibration, and Cognitive Overload metrics, and its rankings align with human judgments, outperforming baseline simulators and revealing model performance differences across user knowledge levels.