arXiv AI

Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation

The paper presents a unified framework for proactive service agents, defining proactivity as an agent’s ability to infer service opportunities from incomplete signals and decide whether to remain silent, ask, assist, or act. It models this as a partially observable sequential decision process constrained by authorization and risk, integrating timing, content, and delivery into a single structured action. The authors categorize existing methods along a decision pipeline—state and need estimation, intervention gating, action construction, and feedback adaptation—and propose standardized metrics for evaluating triggering, timing, calibration, user burden, safety, and policy value across diverse interaction modalities.

arXiv AI
Aug 24

Why2Speak: Faithful Reasoning for Abstaining Action Policies

The paper investigates how agentic systems decide between acting and abstaining, focusing on the fidelity of their reasoning explanations. Using Qwen3‑8B in a multi‑party conversation setting, the authors compare direct decision policies, reasoning policies, supervised fine‑tuning, and reinforcement learning, finding a trade‑off: strong direct policies yield higher performance but no traceable reasoning, while reasoning policies provide an audit trail at the cost of lower recall. The study also uncovers that exposing reasoning can alter the agent’s policy and that common faithfulness metrics may overstate the alignment between reasoning and decisions.

By Shreya Mendi, Brinnae Bent
arXiv AI
Sep 30

Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

The paper introduces a framework for designing proactive large‑language‑model agents, centered on three joint principles—Task Capability, Temporal Allocation, and Trust—alongside a five‑dimensional design space. It proposes PROACTIVITY‑GYM, a simulation testbed for evaluating proactive assistance across multi‑day scenarios, and presents empirical findings that highlight performance gaps and the importance of aligning interventions with user trust. Human studies show that misaligned interventions can sharply reduce trust, even when outcomes are correct, underscoring the need for careful joint optimization of the 3T principles.

By Jio Oh, Seunghyun Do, Young-Jun Lee, Steven Euijong Whang, Dongyeop Kang
arXiv AI
5d ago

PRISM: Festina Lente Proactivity -- Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents

PRISM is a framework for proactive agents that decides when to intervene by using a cost‑sensitive decision gate and a dual‑process reasoning architecture. The agent only engages a resource‑intensive slow reasoning mode near the decision boundary, guided by a calibrated probability of user acceptance and asymmetric costs for missed help versus false alarms. Training employs gate‑aligned distillation, allowing the response policy to be decoupled from the gate for tunable, auditable control, and on the ProactiveBench benchmark PRISM reduces false alarms by 22.78% while improving F1 by 20.14% over strong baselines.

By Yuxuan Fu, Xiaoyu Tan, Teqi Hao, Chen Zhan, Xihe Qiu
arXiv AI
Oct 16

StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning

StepSafe introduces a training framework called ASPO that focuses on step‑level timely intervention in agentic systems. It aligns decisions with their respective prefixes, rewards actions that fit the current stage, and uses adaptive feedback self‑distillation to credit reasoning. The resulting StepSafe guardrail achieves state‑of‑the‑art performance on five agent‑safety benchmarks, improving the safety‑utility Pareto curve with a Youden’s J of 53.0 for a 4B model.

By Siyi Gu, Kunhan Gao, Chao Peter Yang, Sukanta Ganguly, Rex Ying, Arman Cohan
arXiv AI
5d ago

Think Before You Act: Decision Gate for LLM Agents

The paper introduces JevAct, a runtime verification framework that intercepts selected actions of large language model (LLM) agents before execution. JevAct compiles domain policies into executable state checks and uses a prefill-only decision model to provide probability‑based judgments, enabling feedback‑driven correction without modifying the underlying agent. Evaluations on eleven LLM agents across Airline, Telecom, and BFCL benchmarks show that JevAct improves agent reliability, especially on Telecom and BFCL tasks.

By Iftach Shoham, Yuval David, Offer Akrabi, Erez Bilgory, Ido Levy, Sami Marreed, Sergey Zeltyn, Segev Shlomov
arXiv AI
Aug 24

Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning

Consilience is an inference‑time orchestration framework that steers and certifies communication among multi‑agent large language models in hidden‑profile settings. It summarizes each discussion turn with a compact state of uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects a communication intervention (challenge, clarify, seek evidence, or route) and speaker. A round‑wise conformal calibration procedure guarantees that the controller’s proposed action has bounded one‑step regret with high probability, and an acceptance mechanism enforces this guarantee for the executed action. Experiments on HiddenBench‑style tasks show that Consilience improves decision accuracy and communication efficiency over fixed and unstructured protocols, sometimes outperforming a full‑information baseline.

By Abhijith Babu, Ramneet Kaur, Vishal Pramanik, Olivera Kotevska, Nathaniel D. Bastian, Susmit Jha, Sunny Raj, Yanzhao Wu, Sumit Kumar Jha, Anirban Roy
arXiv AI
Oct 1

NAQD Env: A benchmark for selective withdrawal in language agents

The paper introduces NAQD‑Env, a synthetic benchmark designed to test language agents’ ability to selectively withdraw and resume actions when new evidence, permissions, or stop instructions arise. It evaluates models against a deterministic reference policy across eleven dependency families, measuring policy agreement, task value, withdrawal, resumption, and event reporting. Experiments on 350 scenarios show low withdrawal recall, no valid resumption, and limited policy alignment, highlighting the need to treat selective withdrawal as a distinct reliability component.

By Mohamed Abouzahra