The paper investigates when per‑point data‑valuation scores (e.g., Data Shapley, influence, leverage) reduce to simple class balance. It shows that a fixed top‑k selection rule fails when a point’s value depends on what else has been chosen, leading to excess risk in equal latent‑mode models. The authors propose using class balance when labels reveal learner modes and explicit coverage otherwise, demonstrating that class‑balancing per‑point scores can match or outperform generic coverage on standard classification tasks with pretrained features, while label‑free coverage can improve accuracy by up to +0.20.
By Yingjie Bai, Weiming Zhi
arXiv:2610. 29067v1 Announce Type: new Abstract: A selective classifier may be claimed to satisfy a risk-coverage contract if some rule from a predeclared menu (a threshold or a union of $K$ score intervals) attains coverage at least $\kappa$ and error at most $\alpha$.
By Daniel Cie\'slak, Andrzej Czy\.zewski
The paper presents a certified rule that determines whether unseen data should be incorporated into a pretrained model’s context or its weights, based on an exact law for their information value. This law quantifies the code length saved on held‑out queries as the expected bits stored about the query, adjusted by a prior repair and decoding loss, and applies to any model, architecture, dataset, or task. The authors demonstrate that the rule’s ceilings correctly certify choices across diverse models—including transformers, TabPFN, occupancy networks, and stereo networks—and that an anytime‑valid test can reliably select the better approach (e.g., TabPFN over XGBoost or GPT‑2’s cache over fine‑tuning) with controlled error rates.
By Leon Chlon, Julie Huang
arXiv:2610.14028v1 Announce Type: cross
Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...
By Luyang Zhang, Jialu Wang, Bebei Li
arXiv:2607. 18088v1 Announce Type: new Abstract: Standard evaluation of many recognition systems contains distribution shift by construction, since benchmarks place disjoint conditions in the training and test splits.
By Weijia Han, Lisha Qu
The paper introduces First-Order In-Context Shapley, an inference‑only method that evaluates data utility for large language model alignment by computing the first‑order term of an in‑context cooperative game over semantic k‑NN neighborhoods in O(k) forward passes. It projects log‑likelihood shifts onto a directed influence graph, yielding three signals—Exemplifying Advantage, Exemplified Advantage, and One‑of‑a‑Kind Score—to identify high‑utility demonstrations and rare prompts. The approach reduces compute by 19× compared to Monte‑Carlo Shapley, filters 1–5% of records, and improves error detection and reward accuracy in DPO training and benchmark auditing.
whyItMatters":"By enabling scalable, inference‑only data auditing, the method addresses label noise and safety inversions that hinder post‑training alignment of large language models, leading to measurable improvements in model reliability and reward accuracy."
By Yunting Song, Matthew Watson, Moustafa Farid Alzantot, Peter Grabowski, Jun Qin
arXiv:2609.37493v1 Announce Type: cross
Abstract: Serving an answer from a large language model requires deciding when to abstain, yet a verifier's ranking accuracy alone does not determine the error...
By Dongyub Jude Lee, Jungseob Lee, Chanjun Park, Hyeonseok Moon, Heuiseok Lim
The study investigates whether the usefulness of training samples depends on the data itself or on the learner. By fixing selected subsets and varying the learner, the authors show that the crossover point—where easy samples give way to coverage samples—shifts with changes in model width, input resolution, and architecture. This indicates that the boundary between easy and coverage samples is learner‑dependent rather than data‑dependent.
By Yangze Liu, Xiao-Long Yin, Zhongyi Han
Parameter-efficient fine-tuning still leaves a broad space of behavior-changing updates reachable, so a poisoned objective can be represented and optimized. We study an alternative: adaptation constrained to the subspace estimated from a trusted pool of existing task adapters.
arXiv:2607.22258v2 Announce Type: replace
Abstract: Models trained on long-tailed data using standard softmax tend to exhibit higher training error and a larger generalisation gap for classes with fe...
By Yi-Hang Zhu, Rajeev Raman, Shiqi Su, Jianyuan Sun, Xinyu Yang, Nan Xing, Huiyu Zhou
arXiv:2608.21262v1 Announce Type: cross
Abstract: Many machine-learning systems set a threshold at a quantile of a calibration set: conformal predictors that promise 90% coverage by drawing their cut...
By Adam Noonan
arXiv:2506. 20573v4 Announce Type: replace-cross Abstract: Public datasets, crucial for modern machine learning and statistical inference, often contain low-quality or contaminated samples that can harm model performance.
By Kristian Minchev, Dimitar I. Dimitrov, Nikola Konstantinov