arXiv AI

Auditing Chinese Web-scale Corpora via Sampled BPE Token Statistics

arXiv:2608. 10678v1 Announce Type: cross Abstract: Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora.

arXiv AI

Jul 28

CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data

arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.

By Peiguang Li, Yongwei Zhou, Juncheng Diao, Yuchun Fan, Jian Yang, Jianxiao Yang, Zhongda Su, Shuguang Jiao, Xiao Wei, Zhiye Zou, Gan Dong, Zhizhao Zeng, Rongxiang Weng, Jingang Wang, Xunliang Cai

llms efficiency benchmarks

arXiv AI

Sep 1

When Less is More: Understanding When Token Filtering Helps and Fails in AI-generated Text Detection

arXiv:2608.29903v1 Announce Type: cross Abstract: The rapid advancement of large language models (LLMs) has made AI-generated text detection increasingly critical. Existing zero-shot detectors assume...

By Xiaoyang Han, Lvxiaowei Xu, Ming Cai

llms

arXiv AI

Jul 17

Pretraining Data Can Be Poisoned through Computational Propaganda

arXiv:2607. 15267v1 Announce Type: new Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate.

By Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith, David Kohlbrenner, Kyle Lo

llms safety

arXiv AI

Jun 16

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

arXiv:2606. 15396v1 Announce Type: cross Abstract: Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns.

By Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang

llms rag benchmarks safety

arXiv AI

Oct 1

Auditing Agent Actions through Query-Conditioned Attribution

arXiv:2609.33676v2 Announce Type: replace Abstract: LLM agents increasingly take consequential actions through interactions with users, policies, and external tools. Auditing these agents requires au...

By Yifan Liu, Praveen Venkateswaran, Abdulhamid Adebayo, Dong Wang

llms agents benchmarks

arXiv AI

Aug 25

MegaMem: A Retrieval Solution for Ultra-Large Context Windows

arXiv:2608.22137v1 Announce Type: new Abstract: Modern language models and agents increasingly require persistent memory for complete codebases, long interaction histories, and heterogeneous enterpri...

By Xinyuan Song, Bowen Zhu, Hasibul Haque, Liang Zhao

llms rag agents

Hugging Face Trending Papers

Aug 10

Measuring the Tokenization Premium: A Cost Audit for Underserved Language Communities

Large language models are increasingly deployed as general-purpose educational and technical assistance systems, but their underlying infrastructure does not treat languages equally. One underexamined source of disparity is tokenization: semantically equivalent content can require substantially different token counts across languages, affecting API cost, latency, and usable context length before a model is invoked.

llms nlp benchmarks

arXiv Computation and Language

Sep 23

FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

arXiv:2609.25298v1 Announce Type: new Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely...

By Yusser Al Ghussin, Eva Gavaller, Cristina Espa\~na-Bonet, Josef van Genabith, Simon Ostermann

llms nlp benchmarks safety

arXiv Computation and Language

Aug 21

Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection

arXiv:2501. 14728v2 Announce Type: replace-cross Abstract: While generative artificial intelligence (GenAI) models have achieved significant success, their misuse for generating deceptive content raises growing concerns about online information security.

By Zehong Yan, Peng Qi, Wynne Hsu, Mong Li Lee

rag multimodal benchmarks

arXiv AI

Jun 17

The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data

arXiv:2606. 18192v1 Announce Type: new Abstract: As high-quality public web corpora become increasingly exhausted, clean long-context documents have become a scarce and expensive source of training data for large language models (LLMs).

By Nick Bettencourt, Xiaowei Ding, Kay Giesecke

llms computer-vision benchmarks

arXiv AI

Jul 17

In-Place Tokenizer Expansion for Pre-trained LLMs

arXiv:2607. 15232v1 Announce Type: cross Abstract: A tokenizer fixed at the start of pre-training allocates vocabulary in proportion to the pre-training corpus, reflecting the deployment priorities at that time.

By Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera, Simon S. Lee, Paul Pak, Aditya Tadimeti, Tim Seyde, Maxime Labonne, Alexander Amini, Mathias Lechner

llms rag nlp

arXiv AI

Jun 29

Cross-Platform Chinese Offensive Comment Detection via Dual-Threshold Hard Example Mining

arXiv:2606. 27629v1 Announce Type: cross Abstract: Cross-platform deployment of offensive comment detection for Chinese social media suffers performance degradation.

By Ruixing Ren, Junhui Zhao, Fangfang Wang

fine-tuning

Related stories