arXiv:2606. 01806v1 Announce Type: cross Abstract: Small Language Models (SLMs) offer a balance between capability and computational feasibility.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2607. 00510v1 Announce Type: new Abstract: Knowing which training examples drive outputs is fundamental to auditing, correcting, and understanding language models, yet for modern LLMs this remains expensive, approximate, and largely post-hoc.
arXiv:2608. 03071v1 Announce Type: new Abstract: Large language model agents derive much of their capability from tool use.
The paper introduces SuperValid, a framework that generates out-of-distribution, capability-aligned validation data by distilling core concepts from benchmarks and expanding them into diverse, knowledge-rich texts. By focusing on capability-level performance rather than benchmark-specific metrics, SuperValid’s loss correlates strongly and stably with downstream benchmark results across a wide range of models, scales, and training data distributions. This training‑free metric can be computed during training, enabling model selection, early stopping, and scaling decisions without the need for benchmark evaluation.
The paper investigates the often-overlooked scale vectors in large language models, showing that despite their tiny size they are crucial for pre‑training performance. The authors provide theoretical insights that scale vectors mainly aid optimization rather than expressivity, and they analyze how weight decay affects different normalization layers. Building on these findings, they propose lightweight improvements—branch‑specific heterogeneity, better placement, and magnitude‑direction reparameterization—that consistently reduce loss across a range of model sizes and training settings.
arXiv:2604.03592v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) models exhibit striking performance disparities across languages, yet the internal mechanisms driving these gaps rem...
arXiv:2607. 19604v1 Announce Type: cross Abstract: Injecting factual knowledge into large language models (LLMs) reliably and at scale remains an open challenge.