arXiv:2607. 28008v2 Announce Type: replace-cross Abstract: Representation engineering reads and steers capability directions in large language models, yet methods are typically evaluated on paper-specific synthetic data.
By Yanshi Li, Xueru Bai, Shuman Liu, Long Zhang
The paper introduces the Capability-Driven Multimodal Scaling Law, a cross-family framework that predicts vision-language model (VLM) benchmark accuracy from a low-dimensional textual capability score extracted via PCA. By training over 150 VLMs on 34 large language models across seven families, the authors demonstrate that the law accurately extrapolates transfer rates from 8B to 72B‑parameter backbones, predicts full training trajectories, and generalizes to unseen model families. The study also reveals actionable insights, such as certain textual benchmarks negatively correlating with multimodal performance and base LLMs outperforming instruction-tuned counterparts as VLM backbones due to higher absorption rates.
By Ziran Li, Qiang Wang, Zhengyu Chen, Shanglin Lei, Borun Chen, Jingang Wang, Xunliang Cai
The paper introduces a capability manifold, a multidimensional framework that maps downstream capabilities—such as reasoning, retrieval, planning, and adaptation—to pre‑training, post‑training, and test‑time resources via bounded scaling functions. It provides analytical Jacobians to quantify how sensitive each capability is to changes in resources and their interactions. By embedding existing Kaplan‑ and Chinchilla‑type scaling laws and test‑time compute into this manifold, the authors demonstrate that these scaling relationships can be unified as trajectories on a common capability manifold.
By Syed Ali Raza Zaidi, Maryam Hafeez
arXiv:2606. 01806v1 Announce Type: cross Abstract: Small Language Models (SLMs) offer a balance between capability and computational feasibility.
By Sourav Das
Existing machine learning (ML) scaling laws relate predictive loss to compute, model parameters, and data. However, as models are increasingly deployed through agentic harnesses, loss alone is insuffi...
arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.
By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang