The study examines how generative AI has reshaped the public web in under five years, using over 5 million crawlable pages. It finds that AI-generated text has risen sharply but unevenly, with higher prevalence on English-language and .com sites, lower on .gov and .edu, and varying by a page’s centrality in the web graph. The authors employ multiple detection tools and a Bayesian hierarchical model to estimate prevalence while accounting for detector errors.
In less than five years, generative artificial intelligence (AI) has dramatically reshaped the public-facing web. Using a sample of more than 5 million crawlable webpages, we demonstrate that the shar...
DeGenTWeb is a system that identifies websites whose informational prose is largely machine‑generated text (MGT) with minimal human input. By adapting MGT detectors for web pages and aggregating results across sites, the study reports that 20.9% of Common Crawl‑archived sites (Jan–Jul 2026) and 13.6% of Bing‑linked sites (2025) are MGT‑dominant, with a 0.12% mislabeling rate for human‑authored sites. The prevalence of MGT‑dominant sites is increasing over time and many share the same owner.
The study measures Google AI Overviews (AIOs), noting they activate for 13.7% of queries overall and 64.7% of question-form queries, with lower rates for politically sensitive topics. AIOs cite more credible domains than typical search results, yet 30% of cited pages are not shown in the first-page results, indicating a distinct source selection process. Among 98,020 atomic claims, 11.0% lack support from cited pages, mainly due to omissions, and over half of cited pages contain display ads, reducing publisher revenue while Google’s ads remain.
arXiv:2609.15369v1 Announce Type: new Abstract: Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-l...
arXiv:2603.16138v2 Announce Type: replace-cross Abstract: Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these sy...
arXiv:2606. 04906v1 Announce Type: cross Abstract: Although it is generally agreed that AI-generated text poses a broad societal risk, there is no common understanding in the AI-generated text detection literature on what constitutes harmful use.
arXiv:2605.16428v4 Announce Type: replace-cross Abstract: Search engines traditionally complement online content platforms by directing users seeking information to external websites. The emergence o...
arXiv:2511.18354v2 Announce Type: replace-cross Abstract: AI agents, not humans, are becoming the web's primary consumers, yet the web still ships content structured for human browsing, with no nativ...
arXiv:2608. 16824v1 Announce Type: new Abstract: Generative Engine Optimization (GEO) modifies web content to increase its likelihood of being selected and cited by generative search engines.
arXiv:2510. 11560v2 Announce Type: replace-cross Abstract: The advent of LLMs has given rise to generative search, a new search paradigm in which LLMs retrieve information from the web related to a query and synthesize it into a single, coherent response.
The study examines how to allocate web‑crawling resources to gather higher‑quality data for large language model pretraining. By combining document‑level quality scores, web‑graph centrality, and existing crawl coverage, it finds that most variation in quality is between hosts and that sampling a few pages from a host can predict the quality of the rest. The results suggest that depth‑based sampling of high‑scoring hosts yields far more high‑quality pages per fetch than random or centrality‑based strategies, while overall host coverage shows little correlation with quality.
arXiv:2607. 15267v1 Announce Type: new Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate.