arXiv:2510.23160v2 Announce Type: replace Abstract: Data selection for supervised fine-tuning (SFT) often discards low-scoring instruction examples that may retain useful supervision. We propose a ru...
arXiv:2601. 22146v2 Announce Type: replace-cross Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data.
arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.
The paper investigates how the quality of instructions used to generate response pairs affects preference learning for language models. It shows that low‑quality or ambiguous instructions limit the range of response quality, weakening preference signals, and introduces an instruction‑refinement pipeline that improves data quality without discarding examples. Experiments across models and benchmarks demonstrate that refining instructions leads to better alignment and complements other data‑improvement methods.
The paper introduces soft‑prompt tuning as a lightweight, automated method for evaluating large language models (LLMs). By optimizing only ten soft‑prompt vectors—about 0.0006% of a 7B model’s parameters—within 150 steps, the approach adapts models to specific benchmark formats, improving format compliance while preserving knowledge accuracy. Experiments across seven models and eleven datasets show that this method matches strong evaluation‑harness baselines, enhances evaluation signals on challenging datasets, and yields cleaner outputs that align better with exact‑match scoring and independent LLM judges.
The paper investigates how the output format of instruction‑tuned models confounds assessments of data quality and model capability. By analyzing gradient signatures across multiple tasks, interfaces, and model families, the authors demonstrate that the interface can mask or exaggerate performance signals, with certain skills appearing only under specific output formats. They show that the interface‑varying residual is not noise but captures task‑specific information, and that correcting a single generation budget can flip the measured effect of fine‑tuning on a benchmark. "whyItMatters":"The study highlights that current evaluation practices may misrepresent a model’s true capabilities because they rely on the output interface rather than the underlying content, potentially leading to misleading conclusions about data quality and model performance."
The paper introduces CoDIT, a contrastive decoding technique that separates instruction-following behavior from pre-trained world knowledge in large language models. By generating responses that emphasize post-training instruction capabilities while suppressing shared pre-trained knowledge, CoDIT creates instruction-tuning datasets that lead to consistently better model performance than directly generated responses or existing public datasets. The authors also provide theoretical and empirical evidence that CoDIT effectively distills instruction-tuning knowledge from model parameters into text, facilitating cross-architecture transfer.
CriticGen introduces a generation‑aware evaluation framework that generates sample‑specific evaluation dimensions and scoring criteria across categories such as subjective, objective, and self‑derived constraints. These dynamic rubrics produce a score, reason, executable refinement suggestion, and a refined answer, enabling models to diagnose and target flaws in their responses. Experiments show significant gains in rubric quality, score correlation, and actionable feedback, with 73.17% of answers improved and a 93.28% non‑degradation rate.
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).
LuxIT is a monolingual instruction‑tuning dataset for Luxembourgish, created by synthesizing instruction‑answer pairs from native texts using the DeepSeek‑R1‑0528 model and a quality‑assurance LLM‑as‑judge process. The resulting 227,507 high‑quality pairs were used to fine‑tune 14 LLMs (≤15 B parameters), yielding an average accuracy increase of +5.37 percentage points on standardized Luxembourgish proficiency exams and improvements in macro‑averaged F1 on nine of the fourteen downstream NLP tasks. These findings demonstrate that synthetic monolingual data can effectively enhance LLM performance in low‑resource languages and reveal the complex relationship between exam performance and practical NLP gains.
arXiv:2609.01244v1 Announce Type: new Abstract: Every supervised fine-tuning run forces the same chain of decisions, such as learning rate, batch size, LoRA or full fine-tuning, how many epochs, whic...