← Back to all news
arXiv Computation and Language By Qingjie Zhang, Di Wang, Haoting Qian, Liu Yan, Tianwei Zhang, Ke Xu, Qi Li, Minlie Huang, Hewu Li, Han Qiu

Speculating LLMs' Chinese Training Data Pollution from Their Tokens

Read the original on arXiv Computation and Language →

The paper defines and classifies "Polluted Chinese" (PoC) tokens—Chinese tokens in GPT vocabularies that correspond to pornographic or gambling content. It develops a detector by fine‑tuning an LLM to label such tokens using semantics and search‑engine references, then uses token appearance to speculate on training‑data contamination. Experiments across GPT and 23 other LLMs show widespread PoC tokens, with GPT’s vocabulary containing over 40% long Chinese tokens linked to porn or gambling, and the method is validated on datasets like C4 and Pile.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Aug 27

On the Idiosyncrasies of the Mandarin Chinese Classifier System

The paper introduces an information‑theoretic method to quantify idiosyncrasy in Mandarin Chinese classifiers, measuring how much knowing noun semantics reduces uncertainty in classifier choice. Using the Chinese Gigaword corpus, the authors compute mutual information between classifier distributions and other linguistic variables. They find that, overall, classifier choice is not fully idiosyncratic; shape nouns, in particular, reduce uncertainty the most, while other semantic classes show no clear trend.

By Shijia Liu, Hongyuan Mei, Adina Williams, Ryan Cotterell
More like this →
arXiv AI
Oct 28

DeGenTWeb: A First Look at MGT-dominant Websites

DeGenTWeb is a system that identifies websites whose informational prose is largely machine‑generated text (MGT) with minimal human input. By adapting MGT detectors for web pages and aggregating results across sites, the study reports that 20.9% of Common Crawl‑archived sites (Jan–Jul 2026) and 13.6% of Bing‑linked sites (2025) are MGT‑dominant, with a 0.12% mislabeling rate for human‑authored sites. The prevalence of MGT‑dominant sites is increasing over time and many share the same owner.

By Sichang Steven He, Calvin Ardi, Ramesh Govindan, Harsha V. Madhyastha
llms
More like this →