arXiv Computation and Language

Repeated Queries Exhaust an LLM's Brand Recommendations but Not Its Sources

The study examines how repeated identical buying questions affect the brand recommendations of large language models (LLMs) with and without web‑search retrieval. Across 300 question‑engine cells, five engines that did not use web search continued to add new, previously unseen brands up to run 15, while the single retrieval‑enabled engine’s list plateaued earlier. Domain citations continued to grow throughout the runs, indicating that LLMs keep accumulating source diversity even as brand lists stabilize.

arXiv Machine Learning
Sep 25

Who Owns the AI Recommendation? A Multi-Industry Empirical Map of Brand Category Ownership Across Large Language Models

The study examines how large language models (LLMs) like GPT‑5.2, Gemini 3 Flash, and Perplexity sonar‑pro recommend brands across five industries. Using 50 brands and 250 queries repeated five times, the authors measured brand inclusion, recommendation share, competitive vacuum, and co‑mention asymmetry, finding that most queries mention at least one brand and that vacuum prevalence remained stable between February and September 2026. The analysis shows strong cross‑date consistency in recommendation patterns and no emergent clustering of brand mentions, though co‑mention structures deviate from null expectations.

By Dmitrij \.Zatuchin
arXiv AI
Nov 6

The Popular Get More Popular: Auditing Brand Bias in Generative Search

The paper audits brand bias in generative search, focusing on two dimensions: brand popularity bias and brand familiarity bias. It finds that dominant brands appear 1.4x to 2.8x more often than average brands in commercial queries, and that unfamiliar brand signatures significantly lower recommendation rank while fact-preserving rewrites have little effect. These results show that generative search engines reinforce a rich‑get‑richer dynamic, favoring established market leaders and creating a parametric barrier to entry that standard content optimization cannot overcome.

By Giorgio Monaco, Jacopo Lenti, Francesco Pierri
arXiv Computation and Language
Sep 25

Measuring Brand and Source Discovery under Repeated LLM Queries: A Finite-Sample Audit

The study audits large language model (LLM) outputs by measuring how well repeated queries recover a collected set of responses versus the full set of possible outputs. Using sample-based rarefaction on 4,500 responses from 50 buying questions across six configurations, the authors find historical-dictionary median recovery rates between 92.6% and 95.2%, which drop to 89.5%–94.7% after re‑adjudicating all candidate strings. Additional analyses with Gemini 3.1 Pro annotations and matched roster data confirm that recovery percentages vary with extraction methods, question selection, and the finite reference collection, underscoring the need for explicit measurement definitions and sensitivity analyses in LLM audits.

By Dmitrij \.Zatuchin
arXiv AI
Sep 30

AX is the New AEO

arXiv:2609.34951v2 Announce Type: replace Abstract: In 2023, AI models answered from training data and hallucinated when it ran out, and businesses were told to seed that knowledge. Models' training...

By Ido Finder, Assaf Elovic, Gad Shalev, Liad Yosef
arXiv AI
Oct 27

Folding Pooled Usage Examples into the Tool Index Lowers Retrieval of Unused Tools

The paper investigates how adding past usage examples of tools to a shared tool index—called folding—affects the retrieval of tools that an AI agent has never used. While folding improves retrieval for tools with examples, it significantly reduces the likelihood that an unused tool appears in the top five results on two of three evaluated catalogs, dropping from 73% to 40% in one benchmark. The authors propose a new scoring method, ZoE, which separates documentation and usage scores, and show that ZoE retrieves unused tools more often than a simple router or a constant‑boost rule, suggesting that evaluations of pooled agent experience should distinguish between used and unused tools.

By Abhijit Chakraborty
arXiv AI
Oct 5

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact

The study measures Google AI Overviews (AIOs), noting they activate for 13.7% of queries overall and 64.7% of question-form queries, with lower rates for politically sensitive topics. AIOs cite more credible domains than typical search results, yet 30% of cited pages are not shown in the first-page results, indicating a distinct source selection process. Among 98,020 atomic claims, 11.0% lack support from cited pages, mainly due to omissions, and over half of cited pages contain display ads, reducing publisher revenue while Google’s ads remain.

By Haofei Xu, Umar Iqbal, Jacob M. Montgomery
arXiv Computation and Language
Nov 2

Completeness and Cost of Retrieval Structures for Multi-Tool Requests in LLM Agents

The paper evaluates six retrieval structures for multi‑tool requests in LLM agents, measuring how many required tools appear in the top‑k results and the associated LLM calls, tokens, and latency. Across two testbeds—345 AppWorld queries and 103 MCP‑Bench tasks—none of the untrained, single‑LLM‑call structures completes more than 10% of AppWorld queries at k = 16. A trained set‑level retriever and an LLM‑navigated hierarchy achieve higher completeness (up to 58% on MCP‑Bench) but at different costs and failure modes, with complete retrieval sets leading to 100% task success in a downstream AppWorld agent test.

By Shuyang Zhang