← Back to all news
arXiv Computation and Language By Hayden Moore, Abhinav Verma, Anshuman Dutt

Tracing Recurrent and Emergent Hallucination Patterns in Decomposed Text-to-SQL Generation

Read the original on arXiv Computation and Language →

The paper introduces a drill‑down evaluation framework that breaks down Text‑to‑SQL queries into simpler executable sub‑queries and natural‑language questions, enabling analysis of hallucination patterns along the generation path. It distinguishes recurrent hallucinations—those appearing in both early and final steps—from emergent hallucinations that only surface at the end. Experiments on five large language models and an expanded BIRD‑mini benchmark reveal varying recurrence and emergence probabilities across categories, with conversational history generally increasing recurrence and decreasing emergence.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jul 16

Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs

arXiv:2601. 02023v2 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) increasingly utilize massive context windows as working memory for autonomous tasks, their reliability fluctuates significantly depending on how information is distributed in real-world corpora.

By Amirali Ebrahimzadeh, Seyyed M. Salili
llmsagentsbenchmarkssafety
More like this →
arXiv AI
Aug 11

Unified Hallucination Fuzzing for Multimodal Large Language Models

arXiv:2608. 07525v1 Announce Type: cross Abstract: Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs), severely limiting their reliability in high-stakes applications.

By Pengfei Zhou, Jiajun Song, Zhiwei Tang, Yixing Ma, Xiaopeng Peng, Donghui Si, Yuhang Xu, Huiqi Song, Yiyuan Miao, Yichen Qian, Weihua Chen, Wangbo Zhao, Bohan Zhuang, Jiasheng Tang, Yang You
llmsreinforcement-learningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 16

DialFork: Going Beyond Factuality to Evaluate Procedural Hallucination in Conversational Agents

DialFork is a new benchmark that evaluates procedural hallucination in conversational agents, a failure mode where agents produce fluent, factually plausible utterances but deviate from the intended dialogue process. The benchmark consists of 36,278 dialogues across 24 tasks in 12 domains, created by forking conformant dialogues and injecting one of four hallucination types, with validation by 30 human annotators. Three evaluation tasks are defined—monitoring overall deviation, procedural retrieval of a single dialogue, and real‑time detection of off‑track turns—using dialog‑flow‑aware metrics FlowKL and FlowScore, which outperform baseline semantic similarity metrics.

By Sergio Burdisso, Esa\'u Villatoro-Tello, Dairazalia S\'anchez-Cort\'es, Yanis Labrak, Jordi Luque, Milos Cernak, Petr Motlicek
llmsagentsbenchmarkssafety
More like this →