The paper introduces DualSFT, a one-shot algorithm that simultaneously selects a parameter mask and a data subset for fine‑tuning large language models by using a shared gradient‑interaction matrix. By casting both selections as bilevel problems under a common validation objective, the authors derive a unified scoring rule that aggregates parameter importance and data utility as column‑wise and row‑wise sums of the same matrix. Experiments on 3B‑9B LLMs show that DualSFT improves target‑task performance and stability‑plasticity trade‑offs compared to single‑axis variants and outperforms sequential hybrid baselines under matched budgets.
arXiv:2511. 21056v2 Announce Type: replace Abstract: Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs.
arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.
arXiv:2605.07111v3 Announce Type: replace-cross Abstract: Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater represe...
Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.
arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.
The paper introduces SSR, a simple yet effective data selection strategy for million‑scale multi‑modal instruction‑tuning. SSR uses interaction styles as a proxy for structural diversity and decomposes data quality into capability‑specific scores, forming a rich profile for each sample. By casting selection as submodular coverage maximization, SSR consistently outperforms random sampling across two architecture stacks, multiple data pools, and 14 benchmarks, achieving 99.1% of full‑data performance with only 30% of a 2.6M pool.
The paper introduces TESS, a scalable data‑selection framework that replaces per‑sample weights with a selection network to improve transferability across datasets and model sizes. It identifies instability in existing meta‑learning for training‑data selection (MTS) due to weight suppression and overreliance on easy features, and proposes a Pointwise Value Matching objective to address these issues. Experiments on large language model safety and instruction tuning show strong transfer from subsets to full corpora and from smaller to larger models.
arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.
GroupDPO introduces a memory‑efficient approach to group‑wise direct preference optimization for aligning large language models. By using first‑order linearization with per‑response coefficients, the method decouples samples during backpropagation, dramatically reducing peak memory usage and enabling scalable training with larger groups. Experiments in both offline and online settings show that leveraging multiple responses consistently outperforms single‑pair training, and adding a negative log‑likelihood term on positive responses is essential for performance gains and training stability.
arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.
The paper investigates how the choice of Transformer layers for fine‑tuning affects both task performance and the preservation of pre‑trained capabilities. By measuring layer sensitivity with a forward‑only input–output cosine similarity metric, the authors identify layers that are more suitable for adaptation. They introduce Layer‑Selective LoRA (LS‑LoRA), which trains adapters only in low‑similarity layers, and demonstrate that this approach improves target‑task results while better retaining commonsense reasoning compared to standard all‑layer LoRA.
As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Experts (MoE) models. This...