Adapting English Quality Classifiers for Multilingual LLM Pretraining Data Selection

The paper proposes a multilingual adaptation of an English quality classifier to select high‑quality training data for large language model pretraining across more than 100 languages. The method trains a small multi‑layer perceptron on Transformer embeddings, using machine‑translated text scored by the English classifier as labels. Experiments on 1B, 3B, and 8B parameter models show that the adapted classifier preserves benchmark performance and regional knowledge, and it generalizes to unseen languages by learning the original scoring criteria.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• rag
• benchmarks