The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2510.23160v2 Announce Type: replace Abstract: Data selection for supervised fine-tuning (SFT) often discards low-scoring instruction examples that may retain useful supervision. We propose a ru...
arXiv:2601. 22146v2 Announce Type: replace-cross Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data.
arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.
The paper investigates how the quality of instructions used to generate response pairs affects preference learning for language models. It shows that low‑quality or ambiguous instructions limit the range of response quality, weakening preference signals, and introduces an instruction‑refinement pipeline that improves data quality without discarding examples. Experiments across models and benchmarks demonstrate that refining instructions leads to better alignment and complements other data‑improvement methods.
The paper introduces soft‑prompt tuning as a lightweight, automated method for evaluating large language models (LLMs). By optimizing only ten soft‑prompt vectors—about 0.0006% of a 7B model’s parameters—within 150 steps, the approach adapts models to specific benchmark formats, improving format compliance while preserving knowledge accuracy. Experiments across seven models and eleven datasets show that this method matches strong evaluation‑harness baselines, enhances evaluation signals on challenging datasets, and yields cleaner outputs that align better with exact‑match scoring and independent LLM judges.
The paper investigates how the output format of instruction‑tuned models confounds assessments of data quality and model capability. By analyzing gradient signatures across multiple tasks, interfaces, and model families, the authors demonstrate that the interface can mask or exaggerate performance signals, with certain skills appearing only under specific output formats. They show that the interface‑varying residual is not noise but captures task‑specific information, and that correcting a single generation budget can flip the measured effect of fine‑tuning on a benchmark. "whyItMatters":"The study highlights that current evaluation practices may misrepresent a model’s true capabilities because they rely on the output interface rather than the underlying content, potentially leading to misleading conclusions about data quality and model performance."