arXiv AI

The Default Trap: Rethinking Plan Evaluation in Tool-Using LLM Agents

The paper investigates how large language model agents that use tools respond to changes in plan priorities versus default plan removal, a phenomenon termed the "default trap." Experiments across 3,200 decision windows on Retail, Airline, and AgentDojo tasks show that switching priorities strongly redirects model choices, while removing a default plan yields weaker responsiveness. Additional studies reveal that the order of account lists and the presence of extra text significantly influence default target selection and priority effects, with overall task success varying from -19.4 to +8.3 points relative to no plan.

arXiv AI
3d ago

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

Persona prompting is widely used in LLM user simulation, yet it assumes that specifying a target user attribute will shift only that attribute while leaving others unchanged. Across multiple models, the study finds that changing a target attribute also alters unspecified non‑target choices, creating a "neutrality gap" that persists even when no preference is declared. The authors show that blocking query‑time attention reduces non‑target drift, but this also weakens target efficacy, highlighting a central challenge in achieving selective control.

By Jiashen Ren, Wenlin Zhang, Bohan Zhang, Xiaopeng Li, Zichuan Fu, Wanyu Wang, Junyi Li, Xiangyu Zhao
arXiv AI
Sep 18

How Do Agent Harnesses Create Value? Planning Information and Release Control in Stateful LLM Agents

The paper investigates how agent harnesses—specifically planning guidance, execution organization, and completion verification—affect performance in retail and airline pilot tasks. By comparing fixed, task‑specific plans to shuffled policy text of equal length, the study finds that fixed plans improve success rates by about 7 percentage points, especially on complex tasks. A read‑only verifier rejects a majority of invalid episodes while incurring minimal cost, and its impact varies with the penalty for erroneous acceptance, often matching the full planning‑plus‑verification benefit at a lower cost.

By Yukun Zhang, Kemu Xu, Yishen Chen
arXiv AI
6d ago

From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings

The paper introduces Ask2Decide, a benchmark of 43 planning tasks designed to test whether large language model (LLM) agents can identify and request missing decision‑relevant information from humans. It collects 79 interaction sessions from 19 participants, showing that human replies often omit critical facts, and uses these data to calibrate a simulated operator. Evaluation of four state‑of‑the‑art LLMs reveals that their initial plans are infeasible on 58‑98% of tasks when information is missing, and that agents only slightly outperform random guessing in distinguishing useful from irrelevant facts.

By Yuhan Tang, Ke Fan, Xinkai Zou, Baichuan Mo, Kangxin Cui, Ao Qu, Ou Zheng, Jiangbo Yu, Haris N. Koutsopoulos, Jinhua Zhao, Yuan Zhou
arXiv Machine Learning
Oct 22

Second Thoughts: Exploring and Planning through Comparative Imagination in World Models

The paper introduces Sibling Disagreement in Imagination (SDI), a method that uses a shared actor-based reference to guide exploration and planning in world models. SDI compares action sequences imagined from the same state, scaling exploration signals and action acceptance based on the variability of the actor’s own rollouts. Experiments on Meta-World, Adroit, and DM‑Control show SDI outperforms baselines on most tasks and improves recovery from actor failures while preserving successful behavior.

By Hieu Tran, Hong Yu
arXiv AI
3d ago

SwarmSafe-Delta: Isolating Message Effects on LLM Agent Safety

SwarmSafe-Delta is a paired executable protocol designed to isolate the impact of upstream messages on the safety of large language model (LLM) agents. By freezing upstream computation and varying real, hidden, and placebo message delivery, the protocol evaluates a single committed action against deterministic benign and risky policy states within preassigned budgets. Experiments across four models show that real message delivery increases risky-world violations and improves benign task success, while hidden delivery reduces risky outcomes, demonstrating that communication can both enhance utility and affect safety depending on message content and delivery method.

By Junru Zhu
arXiv Machine Learning
Oct 8

Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation

The study investigates whether adaptive layer-skipping and repetition programs truly improve language‑model inference or merely exploit input‑blind controls. Using 32 such programs on Qwen3‑4B‑Base and Llama‑3.1‑8B across 4,413 multiple‑choice items, the authors compare real program gains to those from input‑blind perturbations and fixed letter offsets. Results show that input‑blind controls can produce comparable or even larger headroom, and that real programs only outperform controls in specific menu configurations, indicating that the apparent benefits may not be due to the actual layer computation choices.

By Yibei Guo, Rui Liu