The paper defines and classifies "Polluted Chinese" (PoC) tokens—Chinese tokens in GPT vocabularies that correspond to pornographic or gambling content. It develops a detector by fine‑tuning an LLM to label such tokens using semantics and search‑engine references, then uses token appearance to speculate on training‑data contamination. Experiments across GPT and 23 other LLMs show widespread PoC tokens, with GPT’s vocabulary containing over 40% long Chinese tokens linked to porn or gambling, and the method is validated on datasets like C4 and Pile.
arXiv:2602. 18733v2 Announce Type: replace Abstract: Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance.
arXiv:2604. 14180v2 Announce Type: replace-cross Abstract: We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.
arXiv:2608. 10678v1 Announce Type: cross Abstract: Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora.
The paper introduces an information‑theoretic method to quantify idiosyncrasy in Mandarin Chinese classifiers, measuring how much knowing noun semantics reduces uncertainty in classifier choice. Using the Chinese Gigaword corpus, the authors compute mutual information between classifier distributions and other linguistic variables. They find that, overall, classifier choice is not fully idiosyncratic; shape nouns, in particular, reduce uncertainty the most, while other semantic classes show no clear trend.
arXiv:2610.04898v2 Announce Type: replace-cross Abstract: This study analyzes the predictive power of LM-derived, token-level surprisal on Mandarin Chinese reading times. We first propose the Shortes...
DeGenTWeb is a system that identifies websites whose informational prose is largely machine‑generated text (MGT) with minimal human input. By adapting MGT detectors for web pages and aggregating results across sites, the study reports that 20.9% of Common Crawl‑archived sites (Jan–Jul 2026) and 13.6% of Bing‑linked sites (2025) are MGT‑dominant, with a 0.12% mislabeling rate for human‑authored sites. The prevalence of MGT‑dominant sites is increasing over time and many share the same owner.
arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.
The paper shows that HuggingFace’s ByteLevel pre‑tokenizer, which treats a word as a sequence of Unicode letters, splits abugida scripts at every vowel sign, creating a training‑free lower bound on tokenizer fertility. Across 26 languages, all 17 abugidas exhibit increased token counts (up to 9×), while Latin, Cyrillic, Hangul, and Han remain unchanged. The authors demonstrate that correcting the character class reduces Nepali token counts, improves model performance, and that this issue is widespread in popular HuggingFace models.
arXiv:2506. 15138v2 Announce Type: replace-cross Abstract: Tokenization directly affects the inference efficiency of large language models, since fragmented tokenization increases sequence length and generation cost.
arXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across lang...
The paper introduces C$^2$TU, a training‑free and prompt‑free method for uncovering Chinese cloaked toxic content. It uses substring matching with a Chinese homophone graph and toxic lexicon to spot candidate toxic words, then filters non‑toxic candidates and corrects cloaks using two variants—one based on BERT and another on LLMs that overcome auto‑regressive limitations. Experiments on two Chinese toxic datasets show C$^2$TU outperforms the best competitor by up to 71% in F1 score and 35% in accuracy.
arXiv:2604. 01904v3 Announce Type: replace-cross Abstract: Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.