arXiv Computation and Language

Answer Bubbles: Information Exposure in AI-Mediated Search

arXiv AI
Oct 5

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact

The study measures Google AI Overviews (AIOs), noting they activate for 13.7% of queries overall and 64.7% of question-form queries, with lower rates for politically sensitive topics. AIOs cite more credible domains than typical search results, yet 30% of cited pages are not shown in the first-page results, indicating a distinct source selection process. Among 98,020 atomic claims, 11.0% lack support from cited pages, mainly due to omissions, and over half of cited pages contain display ads, reducing publisher revenue while Google’s ads remain.

By Haofei Xu, Umar Iqbal, Jacob M. Montgomery
arXiv Computation and Language
Sep 2

Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries

arXiv:2609.00319v1 Announce Type: cross Abstract: Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compos...

By Phuong Anh Nguyen, Jill Noorily, Matthew Flathers, Haruka Notsu, Laura Ospina-Pinillos, Tommy Nguyen, Samantha Clark, Aoife Keane, Grace Thompson, John Torous
arXiv AI
Jun 2

Characterizing Web Search in The Age of Generative AI

arXiv:2510. 11560v2 Announce Type: replace-cross Abstract: The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response.

By Elisabeth Kirsten, Jost Grosse Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, Muhammad Bilal Zafar
arXiv AI
Sep 24

Query Implied Generative Engine Optimization

The paper "Query Implied Generative Engine Optimization" introduces QI‑GEO, a method that infers user intent directly from documents to enhance visibility in Generative Search Engines. By approximating a document’s intent space, QI‑GEO identifies missing yet relevant content, improving objective scores by up to 15.9% and subjective scores by up to 17.6% on GEO‑Bench datasets. The approach yields nearly twice as many citation gains as losses, demonstrating that document‑derived intent approximations can boost content visibility without explicit query inputs.

By Shilpa Ramakrishna, William B. Andreopoulos
arXiv Computation and Language
4d ago

How Do LLM-Generated Texts Impact Term-Based Retrieval Models?

The paper examines how large language model (LLM)-generated texts affect term-based retrieval models such as BM25. It finds that LLM outputs have smoother high-frequency and steeper low-frequency Zipf slopes, higher term specificity, and greater document-level diversity, traits that align with LLMs’ focus on reader experience. The study shows that term-based models prioritize documents whose term distributions match the queries, rather than showing an inherent bias toward the source of the text.

By Wei Huang, Keping Bi, Yinqiong Cai, Wei Chen, Jiafeng Guo, Xueqi Cheng
arXiv AI
4d ago

Retrieved but Not Read: Understanding Why Search-Augmented LLMs Fail to Use Available Evidence

The paper investigates why large language models (LLMs) sometimes give outdated answers even when recent evidence is available through web search. By creating temporal facts that changed after the models’ training cutoffs, the authors evaluate a proprietary model with deployed search and six open-weight models with controlled retrieval. They find that models often rely on search snippets without opening source pages, leading to more outdated answers and unfaithful citations, and that even when retrieved evidence influences internal predictions, the final answer may still favor outdated information.

By Owen Walters, Lihu Chen, Francesca Toni
arXiv AI
Sep 17

Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

The paper introduces REASONS, a benchmark comprising 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution by large language models. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to assess the trade-off between reliability and responsiveness. Experiments on proprietary and open-source LLMs under various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but may increase abstention, while retrieval-augmented variants often maintain near-zero abstention. Human evaluation reveals a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur