Hugging Face Trending Papers
Geometric Calibration of Supervised Fine-Tuning
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
5d ago
Geometric Calibration of Supervised Fine-Tuning
arXiv:2610.18634v1 Announce Type: cross Abstract: Recent studies investigate the weight geometry through its spectral structure, which characterizes how the weights transform different input directio...
arXiv AI
Oct 7
Harmful SFT Leaves a Continuous Trace in LLM Checkpoint Updates
arXiv:2610.07518v1 Announce Type: cross Abstract: Safety auditing of post-trained large language models typically relies on model behavior, requiring model execution and depending on the coverage of...
arXiv Machine Learning
Oct 12
A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges
The paper presents a comprehensive framework for comparing parameter‑efficient fine‑tuning (PEFT) methods by treating each as a smooth map from a small parameter space into weight space, with a pretrained checkpoint as the base point. It identifies three key invariants—image near the base point, fibre dynamics, and base change—that explain expressivity, training dynamics, and initialization behavior. The authors prove numerous results about common PEFT techniques (e.g., LoRA, Householder adaptation) and classify 382 methods from 138 papers using seven coordinate descriptors.
arXiv AI
Sep 16
ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training
ReDraft is a reference‑driven revision method for continual post‑training of large multimodal language models. It uses the model’s own incorrect outputs as references, revises them, verifies the revisions, and fine‑tunes on the accepted ones, thereby combining explicit supervision with policy proximity. On tasks such as Counting, Clock Reading, and Jigsaw, ReDraft outperforms standard supervised fine‑tuning and on‑policy methods, achieving higher target‑task gains while dramatically reducing forgetting.
arXiv AI
Jul 29
Similar Models Learn Differently: Final-Window Pretraining Shapes Post-Training Beyond SFT
arXiv:2607. 25063v1 Announce Type: new Abstract: Developers judge a model checkpoint by how it behaves.
arXiv AI
Aug 24
UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialists
UpgradeBench is a decision‑centric longitudinal benchmark that evaluates how fine‑tuned language‑model specialists should be handled when new base‑model releases occur. It covers four consecutive Qwen releases, a continuation checkpoint, six tasks, two model sizes, and OLMo checkpoints with known training lineage, and examines whether retraining, adapter transfer, or other recovery strategies improve specialist performance. The benchmark reveals that upgrade gains vary by task and release interval, that direct adapter copying is sensitive to pretraining distance, and that teacher relabeling can recover specialists without new annotations. "whyItMatters":"The study provides actionable insights into the cost‑effective management of specialist models across model releases, showing how to balance retraining effort with performance gains."