arXiv:2609.24290v1 Announce Type: new
Abstract: Instruction-tuned LLMs faced with underspecified queries often commit to a single interpretation rather than ask for clarification, producing confident...
By Yunxiang Li, Xixin Wu, Helen Meng
arXiv:2605.25831v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) define a distribution over text, which can be viewed as a probabilistic representation of uncertainty: sampling...
By Joris Baan, Wilker Aziz, Barbara Plank, Raquel Fern\'andez
The paper introduces ClariEdge, a dataset of about 3,900 examples that capture ambiguous or ill‑formed user queries in retrieval‑augmented generation contexts, including negative cases that do not need clarification. A lightweight LoRA fine‑tuned model achieves 96.6% classification accuracy, outperforming larger prompted baselines in both classification and clarification generation. The study highlights the importance of hard negatives for distinguishing truly ambiguous queries and demonstrates that the same model can be applied before and after retrieval in a RAG pipeline.
By Lihong He, Lucian Popa, Marina Danilevsky
The paper presents a tri‑agent framework for evaluating large language models’ question‑clarification abilities. It involves a Question Clarifying Agent that identifies ambiguities and asks follow‑up questions, a Respondent Agent that simulates human replies, and an Evaluator Agent that judges the dialogue using metrics such as ambiguity handling, question quality, dialogue efficiency, language appropriateness, and intent alignment. The authors illustrate the approach with synthetic supply‑chain data and discuss validating the evaluator against human judgments.
By Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra
Large Language Models (LLMs) are increasingly deployed in interactive systems where understanding user intent precisely is paramount. A key capability for such systems is effective question clarificat...
arXiv:2607. 21143v1 Announce Type: cross Abstract: Ambiguous user requests make clarification a sequential decision problem for conversational LLM assistants: they must decide whether to ask, what to ask, when to stop, and when to answer.
By Minh Ngoc Ta, My Anh Tran Nguyen, Duong D. Nguyen, Yuxia Wang, Preslav Nakov
arXiv:2608. 11631v1 Announce Type: new Abstract: In open-domain human-computer interaction scenarios, large language models (LLMs) frequently encounter user queries that are ambiguous or incomplete.
By Kuangzhao Yang, Ziliang Zhao, Zhicheng Dou
arXiv:2511. 10453v4 Announce Type: replace-cross Abstract: Large language models often respond to ambiguous requests by implicitly committing to one interpretation, frustrating users and creating safety risks when that interpretation is wrong.
By Irina Saparina, Mirella Lapata
The paper introduces VagueBench, a benchmark that degrades seed queries into vague variants to evaluate large language models’ ability to ask follow‑up questions when user queries are incomplete. It proposes DialogGRPO, an offline reinforcement learning approach that models multi‑turn elicitation as a dialogue‑state directed acyclic graph, using dense turn‑level rewards to guide question selection. DialogGRPO achieves over 88% success in eliciting missing information on vague queries, improves answer coverage and recovery efficiency, and outperforms prompting and test‑time scaling baselines while generalizing zero‑shot to other dialogue datasets.
By Rongyu Zhang, Mutian Bao, Jinjian Zhang, Yuetian Du, He Xu, Wei Zhou, Xiaojun Zhu, Ming Kong, Jie Liu, Linjian Mo, Qiang Zhu
arXiv:2604. 09497v2 Announce Type: replace-cross Abstract: Accurate evaluation is central to the large language model (LLM) ecosystem, guiding model selection and downstream adoption across diverse use cases.
By Hippolyte Gisserot-Boukhlef, Nicolas Boizard, Emmanuel Malherbe, C\'eline Hudelot, Pierre Colombo
arXiv:2610.08026v1 Announce Type: new
Abstract: In recent years, several methods for detecting when large language models (LLMs) hallucinate have been developed. These methods are often benchmarked w...
By Jorma Valjakka, Juhani Kivim\"aki, Juha Myll\"ari, Jukka K. Nurminen
arXiv:2606. 03135v1 Announce Type: new Abstract: Large Language Model (LLM) agents often operate under underspecified user instructions, where latent uncertainty over user intent leads to erroneous tool actions.
By Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu, Ying Zhao, Zhijiang Guo, Wei Wang