arXiv AI

Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiency

arXiv Computation and Language
Oct 5

Beyond Correctness: Resolving Underspecification in Agentic Text-to-SQL

The paper introduces PlanPool, a method for agentic Text-to-SQL systems to manage clarification questions by maintaining a mutable question pool. By requiring agents to explicitly ask or drop each planned question, PlanPool improves ambiguity coverage and reduces silent failures compared to unconstrained or prompt-based approaches. Experiments on benchmarks derived from BIRD-Interact and Spider show that PlanPool achieves competitive execution accuracy while better handling underspecification.

By Wen-Zhi Li, Yue Gong, Konstantinos Kanellis, Balakrishnan Murali Narayanaswamy
arXiv AI
21h ago

Persistent Retrieval for Conversational Agents: A Study of Retrieval Efficiency

The paper investigates how different retrieval designs affect conversational agents. Three approaches are compared: skill-based retrieval in the main agent, a non‑persistent retrieval subagent that generates findings, and a persistent retrieval subagent that keeps history and returns evidence to the main agent. The persistent design reduces token usage and API cost by 43.3% and 45.0% respectively, cuts response time by 9.1%, and allows the main agent to request corrections from the subagent.

By Karthik Ravula, Sathwik Kowtha, Rishabh Verma, Kala Mehak Jain, Gunjit Mittal
arXiv Computation and Language
Sep 3

A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models

The paper presents a tri‑agent framework for evaluating large language models’ question‑clarification abilities. It involves a Question Clarifying Agent that identifies ambiguities and asks follow‑up questions, a Respondent Agent that simulates human replies, and an Evaluator Agent that judges the dialogue using metrics such as ambiguity handling, question quality, dialogue efficiency, language appropriateness, and intent alignment. The authors illustrate the approach with synthetic supply‑chain data and discuss validating the evaluator against human judgments.

By Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra
arXiv Computation and Language
2d ago

Don't Guess, Ask Wisely: Benchmarking and Training Proactive Elicitation in Open-Domain LLM Conversations

The paper introduces VagueBench, a benchmark that degrades seed queries into vague variants to evaluate large language models’ ability to ask follow‑up questions when user queries are incomplete. It proposes DialogGRPO, an offline reinforcement learning approach that models multi‑turn elicitation as a dialogue‑state directed acyclic graph, using dense turn‑level rewards to guide question selection. DialogGRPO achieves over 88% success in eliciting missing information on vague queries, improves answer coverage and recovery efficiency, and outperforms prompting and test‑time scaling baselines while generalizing zero‑shot to other dialogue datasets.

By Rongyu Zhang, Mutian Bao, Jinjian Zhang, Yuetian Du, He Xu, Wei Zhou, Xiaojun Zhu, Ming Kong, Jie Liu, Linjian Mo, Qiang Zhu