arXiv AI

From Crawled to Published: A Publisher-Controlled Web Architecture for an AI-Native Internet

arXiv AI
Jun 2

Characterizing Web Search in The Age of Generative AI

arXiv:2510. 11560v2 Announce Type: replace-cross Abstract: The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response.

By Elisabeth Kirsten, Jost Grosse Perdekamp, Qinyuan Wu, Mihir Upadhyay, Krishna P. Gummadi, Muhammad Bilal Zafar
arXiv AI
Sep 23

SemDHT: Certified Semantic Discovery for Peer-to-Peer Agent Networks over Exact-Key DHTs

SemDHT introduces a certified semantic index for discovering agent-accessible capabilities over exact-key distributed hash tables (DHTs). It uses a two-layer semantic sketch—coarse cells for grouping descriptors and residual codes for refining candidate selection—allowing providers to publish at a bounded set of derived keys while requesters probe precision keys first. Anchor committees certify descriptor-to-key consistency, enabling efficient, consistent discovery with fewer lookups and reduced publication fan‑out, as demonstrated by high recall and significant speedups over locality‑sensitive hashing in real‑world experiments.

By Taotao Wang, Chonghe Zhao, Shengli Zhang, Soung Chang Liew
arXiv AI
Oct 5

Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact

The study measures Google AI Overviews (AIOs), noting they activate for 13.7% of queries overall and 64.7% of question-form queries, with lower rates for politically sensitive topics. AIOs cite more credible domains than typical search results, yet 30% of cited pages are not shown in the first-page results, indicating a distinct source selection process. Among 98,020 atomic claims, 11.0% lack support from cited pages, mainly due to omissions, and over half of cited pages contain display ads, reducing publisher revenue while Google’s ads remain.

By Haofei Xu, Umar Iqbal, Jacob M. Montgomery
arXiv AI
Oct 12

PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real

PRIME introduces an index-side optimization framework for structured document retrieval, comprising a Semantic Restructuring track that converts long documents into dense, structured sections and a Metadata Synthesis & Hierarchical Retrieval track that extracts semantic anchors and heading structures for efficient coarse-to-fine retrieval. The approach reduces embedding computation by 45.8× and index vector count by 22.4× while maintaining retrieval quality comparable to structure-aware chunking. In full deployment, Semantic Restructuring improves Answer‑F1 by 13.7%, and the study identifies qualitative regimes that influence when index-side optimization is beneficial or detrimental, noting that strong rerankers neutralize these effects.

By Cheuk Hei Chung, Li Liu
arXiv AI
Sep 18

Characterizing Web Search by Conversational LLM Agents: From Search Decisions and Strategies to Results and Responses

The study examines how conversational LLM agents—specifically ChatGPT, Claude, Grok, and DeepSeek—use Web search, combining real user interactions with controlled API experiments. It finds that agents differ in when they decide to search, how they craft queries, and which domains they favor, and that more frequent searching does not always improve answer quality. While most responses are grounded in search results, some claims are unsupported, raising attribution concerns.

By Mahsa Amani, Seungeon Lee, Abhisek Dash, Asmaa El Fraihi, Yunah Jang, Elisabeth Kirsten, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander, Muhammad Bilal Zafar, Soumi Das