arXiv AI

Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement

arXiv:2603. 08924v2 Announce Type: replace-cross Abstract: AI-powered answer engines are inherently non-deterministic: identical queries submitted at different times can produce different responses and cite different sources.

arXiv Computation and Language
Sep 15

CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Search

CITECHOICE is a causal audit that examines how the presentation of documents in an agentic search engine redistributes citation credit. Using 129 everyday‑query transcripts, the study compares structured versus prose renderings of the same source while keeping all other transcript elements fixed. The results show that structured rendering increases the target’s citation count by about half a citation per answer without adding total citations or diminishing competitors’ credit, while also revealing that rank position has a larger effect on citation rates than presentation order alone.

By Sriram Selvam, Anneswa Ghosh
arXiv Computation and Language
Sep 25

Measuring Brand and Source Discovery under Repeated LLM Queries: A Finite-Sample Audit

The study audits large language model (LLM) outputs by measuring how well repeated queries recover a collected set of responses versus the full set of possible outputs. Using sample-based rarefaction on 4,500 responses from 50 buying questions across six configurations, the authors find historical-dictionary median recovery rates between 92.6% and 95.2%, which drop to 89.5%–94.7% after re‑adjudicating all candidate strings. Additional analyses with Gemini 3.1 Pro annotations and matched roster data confirm that recovery percentages vary with extraction methods, question selection, and the finite reference collection, underscoring the need for explicit measurement definitions and sensitivity analyses in LLM audits.

By Dmitrij \.Zatuchin
arXiv AI
Sep 10

Attribution in Scientific Literature: New Benchmark and Methods

The paper introduces REASONS, a benchmark of 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution under different evidence conditions. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to balance reliability and responsiveness. Experiments with proprietary and open-source LLMs across various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but increases abstention, while adversarial metadata can push hallucination rates above 85%. Human evaluation confirms a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur
arXiv AI
Sep 24

Query Implied Generative Engine Optimization

The paper "Query Implied Generative Engine Optimization" introduces QI‑GEO, a method that infers user intent directly from documents to enhance visibility in Generative Search Engines. By approximating a document’s intent space, QI‑GEO identifies missing yet relevant content, improving objective scores by up to 15.9% and subjective scores by up to 17.6% on GEO‑Bench datasets. The approach yields nearly twice as many citation gains as losses, demonstrating that document‑derived intent approximations can boost content visibility without explicit query inputs.

By Shilpa Ramakrishna, William B. Andreopoulos
arXiv AI
Sep 17

Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

The paper introduces REASONS, a benchmark comprising 12,723 sentence-level citation instances across 12 arXiv subject categories, to evaluate scientific citation attribution by large language models. It proposes a dual-metric framework—Abstention Rate (AR) and Hallucination Rate (HR)—to assess the trade-off between reliability and responsiveness. Experiments on proprietary and open-source LLMs under various prompting and retrieval settings show that advanced Retrieval-Augmented Generation (RAG) reduces hallucinations but may increase abstention, while retrieval-augmented variants often maintain near-zero abstention. Human evaluation reveals a high ratio of factual hallucinations to acceptable paraphrases, underscoring the need for systems that can appropriately abstain under uncertainty.

By Deepa Tilwani, Yash Saxena, Seyedali Mohammadi, Ankur Padia, Edward Raff, Amit Sheth, Srinivasan Parthasarathy, Manas Gaur
arXiv AI
Jun 2

Characterizing Web Search in The Age of Generative AI

arXiv:2510. 11560v2 Announce Type: replace-cross Abstract: The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response.

By Elisabeth Kirsten, Jost Grosse Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, Muhammad Bilal Zafar