arXiv:2606. 04401v1 Announce Type: new Abstract: The capabilities of large language models (LLMs) significantly depend on training data drawn from various domains.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2607. 01104v1 Announce Type: cross Abstract: In Large Language Model (LLM) training, data mixing plays a pivotal role in determining model performance.
arXiv:2607. 11052v1 Announce Type: new Abstract: Machine learning progress is often attributed to scaling model size and dataset volume, yet the composition of data can be just as consequential.
The paper introduces Simplex-Lattice Interpolation Merging (SLIM), a method that builds a quadratic surrogate model of aggregate performance over merging coefficients for large language models using a classical mixture design. By evaluating only individual experts and equal-weight pairs, SLIM identifies the surrogate with the fewest measurements needed and then optimizes it without further target-metric evaluations. Experiments on two model architectures show that SLIM accurately predicts unseen multi-expert mixtures and achieves competitive merge performance under limited evaluation budgets, outperforming random designs and regularized fitting.
arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters...
The article proposes treating large language model (LLM) data mixing as a classical mixture experiment, where data domains are components, token shares are proportions, and proxy-training runs serve as design points. Using sparse second‑order Scheffé response‑surface models, the authors construct model‑robust Σ‑optimal designs that efficiently identify optimal data mixtures and reveal strong interaction effects, especially between weak domains and web‑derived text. Empirical results on RegMix show that these designs recover mixture rankings while reducing proxy runs by about 25%, demonstrating that data mixing can be optimized through experimental design rather than solely prediction.
arXiv:2601.16390v2 Announce Type: replace-cross Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...