arXiv AI

Auditing Chinese Web-scale Corpora via Sampled BPE Token Statistics

arXiv:2608. 10678v1 Announce Type: cross Abstract: Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora.

arXiv AI

Jul 28

CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data

arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.

By Peiguang Li, Yongwei Zhou, Juncheng Diao, Yuchun Fan, Jian Yang, Jianxiao Yang, Zhongda Su, Shuguang Jiao, Xiao Wei, Zhiye Zou, Gan Dong, Zhizhao Zeng, Rongxiang Weng, Jingang Wang, Xunliang Cai

llms efficiency benchmarks

arXiv AI

Jul 17

Pretraining Data Can Be Poisoned through Computational Propaganda

arXiv:2607. 15267v1 Announce Type: new Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate.

By Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith, David Kohlbrenner, Kyle Lo

llms safety

arXiv AI

Jun 16

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

arXiv:2606. 15396v1 Announce Type: cross Abstract: Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns.

By Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang

llms rag benchmarks safety

Hugging Face Trending Papers

Aug 10

Measuring the Tokenization Premium: A Cost Audit for Underserved Language Communities

Large language models are increasingly deployed as general-purpose educational and technical assistance systems, but their underlying infrastructure does not treat languages equally. One underexamined source of disparity is tokenization: semantically equivalent content can require substantially different token counts across languages, affecting API cost, latency, and usable context length before a model is invoked.

llms nlp benchmarks

arXiv Computation and Language

2d ago

Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection

arXiv:2501. 14728v2 Announce Type: replace-cross Abstract: While generative artificial intelligence (GenAI) models have achieved significant success, their misuse for generating deceptive content raises growing concerns about online information security.

By Zehong Yan, Peng Qi, Wynne Hsu, Mong Li Lee

rag multimodal benchmarks

arXiv AI

Jun 17

The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data

arXiv:2606. 18192v1 Announce Type: new Abstract: As high-quality public web corpora become increasingly exhausted, clean long-context documents have become a scarce and expensive source of training data for large language models (LLMs).

By Nick Bettencourt, Xiaowei Ding, Kay Giesecke

llms computer-vision benchmarks

arXiv AI

Jul 17

In-Place Tokenizer Expansion for Pre-trained LLMs

arXiv:2607. 15232v1 Announce Type: cross Abstract: A tokenizer fixed at the start of pre-training allocates vocabulary in proportion to the pre-training corpus, reflecting the deployment priorities at that time.

By Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera, Simon S. Lee, Paul Pak, Aditya Tadimeti, Tim Seyde, Maxime Labonne, Alexander Amini, Mathias Lechner

llms rag nlp

arXiv AI

Jun 29

Cross-Platform Chinese Offensive Comment Detection via Dual-Threshold Hard Example Mining

arXiv:2606. 27629v1 Announce Type: cross Abstract: Cross-platform deployment of offensive comment detection for Chinese social media suffers performance degradation.

By Ruixing Ren, Junhui Zhao, Fangfang Wang

fine-tuning

arXiv Machine Learning

Jun 2

MINES: Explainable Anomaly Detection through Web API Invariant Inference

arXiv:2512. 06906v2 Announce Type: replace-cross Abstract: Detecting the anomalies of web applications, important infrastructures for running modern companies and governments, is crucial for providing reliable web services.

By Wenjie Zhang, Yun Lin, Chun Fung Amos Kwok, Xiwen Teoh, Xiaofei Xie, Frank Liauw, Hongyu Zhang, Jin Song Dong

llms benchmarks

arXiv AI

Jun 3

Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation

arXiv:2606. 03128v1 Announce Type: cross Abstract: Smart contracts face critical security challenges that require thorough auditing in decentralized web services.

By Bagus Rakadyanto Oktavianto Putra, Muhamad Risqi Utama Saputra, Widyawan, Guntur Dharma Putra

llms efficiency benchmarks safety

arXiv Machine Learning

Jul 7

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

arXiv:2512. 20757v2 Announce Type: replace-cross Abstract: Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs).

By G\"ul Sena Alt{\i}nta\c{s}, Malikeh Ehghaghi, Brian Lester, Fengyuan Liu, Wanru Zhao, Marco Ciccone, Colin Raffel

llms nlp benchmarks

arXiv AI

Jun 12

One Polluted Page Is Enough: Evaluating Web Content Pollution in Generative Recommenders

arXiv:2606. 13610v1 Announce Type: cross Abstract: Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content.

By Minghao Luo, Liang Chen

llms benchmarks

Related stories