arXiv Computation and Language
IntentDelta: A Human-Constructed Benchmark for Evolving User Intent
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Aug 28
Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory
Behavior2Trip introduces a new task—Behavior‑Aware Travel Planning—where user preferences are inferred from past behavior trajectories rather than explicit instructions. The benchmark contains 11,400 instances from a major Chinese travel platform, each with nearly 40 recorded behaviors across 14 attributes and 5 preference dimensions. A reinforcement‑learning agent, B2T‑Agent, leverages these trajectories, external retrieval tools, and internal memory, outperforming GPT‑4.1 and other baselines on the dataset.
arXiv AI
Aug 17
Act2Intention: A Benchmark For Developing Active Mobile Agents Through Inferring User Intention from GUI Actions
arXiv:2608. 14132v1 Announce Type: cross Abstract: Mobile GUI Agents powered by multimodal large language models (MLLMs) show promise in human-computer intelligence.
Hugging Face Trending Papers
Aug 27
Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory
Behavior2Trip introduces a new task—Behavior‑Aware Travel Planning—where user preferences are inferred directly from past behavior trajectories rather than explicit instructions. The benchmark contains 11,400 Chinese travel‑planning instances, each with an average of 39.8 past behaviors across 14 attributes and 5 preference dimensions. A reinforcement‑learning agent, B2T‑Agent, leveraging behavior trajectories, external retrieval tools, and internal memory, outperforms strong baselines such as GPT‑4.1 on this challenging dataset.
arXiv Computation and Language
Aug 28
INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignment
The paper introduces INTENT-AS-A-TOOL, a method that equips large language models with intent-targeted tools to provide a fine-grained, judge‑free signal of their commitment to specific behaviors during reasoning. By monitoring the probability of calling these intent tools, the authors can track how intent evolves throughout generation, complementing chain‑of‑thought monitoring and expanding post‑hoc labels into dense trajectories. The approach identifies critical steps for online intervention, demonstrating that action preferences are useful for detecting agentic misalignment in autonomous agents.
arXiv AI
Sep 17
RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents
RideWay is a new benchmark that evaluates ride‑hailing language agents not just on task completion but on interaction efficiency. It introduces the Efficiency Utility metric, which penalizes agents for excessive tool calls and user‑facing turns relative to a task‑specific reference effort, with human preferences used to calibrate the penalties. Across 58 tasks and 24 models, the metric shows that extra dialogue is penalized more heavily than extra tool use, and it achieves high accuracy in distinguishing trajectories that differ in turns but struggles when differences are only in tool calls.
Hugging Face Trending Papers
Sep 16
RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents
AI agents are usually evaluated by whether they complete a task. In interactive service settings, a successful agent can still frustrate users by asking repeated questions, performing redundant search...