The study investigates how the composition of data during the mid‑training phase of language models affects performance across multiple domains. Experiments with Qwen3‑8B‑Base on five distinct KOR‑Bench domains show that moderate coverage (10%‑40%) yields the best per‑domain results, and that alignment passes cannot fully close the performance gaps created by mid‑training data choices. Additionally, zero coverage in mid‑training severely degrades accuracy, while a carefully tuned allocation can provide the largest overall pipeline improvement.
By Yunpeng Xu, Kun Zheng
arXiv:2610.17093v1 Announce Type: new
Abstract: A model diff credits post-training with whatever separates a base checkpoint from one post-trained descendant, presuming a rerun would land close by. O...
By Xupeng Chen, Chenqian Le, Jiayi Cheng, Runhao Li, Michael Yue
The paper investigates how per-domain data composition during the mid‑training phase (between pre‑training and alignment) affects model performance. Experiments with Qwen3‑8B‑Base across five KOR‑Bench domains show that a moderate coverage band (10%‑40%) yields the best performance for each domain, and that alignment passes cannot fully close the gaps created by suboptimal mid‑training allocations. Additionally, zero coverage during mid‑training severely degrades accuracy, while a carefully tuned allocation can provide the largest overall pipeline improvement.
arXiv:2609.24194v1 Announce Type: new
Abstract: Evaluation scores used around LLM systems -- including reward models, rerankers, and LLM judges -- can track surface form instead of the quality they c...
By Daein Weon, Dongho Kang
arXiv:2607. 17136v1 Announce Type: cross Abstract: Agentic computer-use RL is reported in single runs, and those numbers mislead.
By Barada Sahu (Cabal AI), Shivesh Pandey (Para AI)
StrictPRMBench is a replayable evaluation benchmark designed to audit fixed‑score PRM aggregation rules. It standardizes mathematical reasoning pools, scorer evidence, and deletion interventions so that different reducers confront identical selection problems. Across seven checkpoints and two benchmarks, the benchmark reveals that product aggregation yields higher deletion‑induced selection instability than geometric mean, with statistically significant differences in most cells.
By Pengfei Zheng