← Back to all news
arXiv Machine Learning By Yuxuan Wang, Guangxiang Zhao, Xinyu Tan, Yahui Han, Xinxiang Yin, Zhonghai Wu, Xiangzheng Zhang, Lin Sun, Tong Yang

Beyond Distribution Matching: Selective Margin Preservation for Supervised Fine-Tuning

Read the original on arXiv Machine Learning →

The paper introduces Selective Rank-Preserving Supervised Fine‑Tuning (SRP‑SFT), a method that applies a hinge‑based soft margin target to preserve selected local non‑target logit margins during teacher‑forced fine‑tuning. SRP‑SFT uses reference‑scaled lower bounds for pairs from the reference’s top‑K non‑target tokens, stopping once the bound is met. Experiments on Qwen3‑4B‑Base and UltraFeedback show that SRP‑SFT improves MATH‑500 and code‑generation scores compared to cross‑entropy and LP‑SFT baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 30

PACT: Pairwise-Anchored Calibrated Tuning for Single-Token Typed Decisions

The paper introduces PACT, a tuning method for single-token typed-decision models that leverages contrastive pair data to add four training terms—difference-in-differences margin, permutation-consistency, evidence-necessity, and ordinal transport cost—without requiring new annotations. PACT achieves comparable accuracy to existing recipes while reducing position bias and ordinal error, and it improves robustness and stability across seeds. The authors provide code, data splits, and trained adapters for reproducibility.

By Yida Lin
safety
More like this →
arXiv Computation and Language
Aug 31

Nested Byte-Level Vocabularies Are Cheap to Deploy and Expensive to Share: A Pre-Registered Negative Result

The paper demonstrates that a byte‑level BPE tokenizer can be sliced to create multiple vocabulary sizes from a single trained model, preserving exact logits while reducing deployed weights by 66%. Experiments on 30 models show that while sliced models match the full model numerically, they underperform fixed‑cap specialists by a few percentage points in bits‑per‑byte. Multi‑cap training improves robustness to typographical noise, suggesting benefits from training across multiple granularities rather than from control tokens alone.

By Christos Koutsiaris
llmsragnlpbenchmarks
More like this →