The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper investigates when per‑point data‑valuation scores (e.g., Data Shapley, influence, leverage) reduce to simple class balance. It shows that a fixed top‑k selection rule fails when a point’s value depends on what else has been chosen, leading to excess risk in equal latent‑mode models. The authors propose using class balance when labels reveal learner modes and explicit coverage otherwise, demonstrating that class‑balancing per‑point scores can match or outperform generic coverage on standard classification tasks with pretrained features, while label‑free coverage can improve accuracy by up to +0.20.
arXiv:2610. 29067v1 Announce Type: new Abstract: A selective classifier may be claimed to satisfy a risk-coverage contract if some rule from a predeclared menu (a threshold or a union of $K$ score intervals) attains coverage at least $\kappa$ and error at most $\alpha$.
The paper presents a certified rule that determines whether unseen data should be incorporated into a pretrained model’s context or its weights, based on an exact law for their information value. This law quantifies the code length saved on held‑out queries as the expected bits stored about the query, adjusted by a prior repair and decoding loss, and applies to any model, architecture, dataset, or task. The authors demonstrate that the rule’s ceilings correctly certify choices across diverse models—including transformers, TabPFN, occupancy networks, and stereo networks—and that an anytime‑valid test can reliably select the better approach (e.g., TabPFN over XGBoost or GPT‑2’s cache over fine‑tuning) with controlled error rates.
arXiv:2610.14028v1 Announce Type: cross Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...
arXiv:2607. 18088v1 Announce Type: new Abstract: Standard evaluation of many recognition systems contains distribution shift by construction, since benchmarks place disjoint conditions in the training and test splits.
The paper introduces First-Order In-Context Shapley, an inference‑only method that evaluates data utility for large language model alignment by computing the first‑order term of an in‑context cooperative game over semantic k‑NN neighborhoods in O(k) forward passes. It projects log‑likelihood shifts onto a directed influence graph, yielding three signals—Exemplifying Advantage, Exemplified Advantage, and One‑of‑a‑Kind Score—to identify high‑utility demonstrations and rare prompts. The approach reduces compute by 19× compared to Monte‑Carlo Shapley, filters 1–5% of records, and improves error detection and reward accuracy in DPO training and benchmark auditing. whyItMatters":"By enabling scalable, inference‑only data auditing, the method addresses label noise and safety inversions that hinder post‑training alignment of large language models, leading to measurable improvements in model reliability and reward accuracy."