arXiv Machine Learning June 9, 2026 By Edward Sun, Dmitrii Troitskii

Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning Updates

Read the original on arXiv Machine Learning →

arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 25

Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration

arXiv:2607.27836v2 Announce Type: replace Abstract: Large language model unlearning is consistently fragile under relearn attacks. On TOFU, fine-tuning on twenty forget examples substantially recover...

By Xiangyu Yin, Jiaxu Liu, Zhen Chen, Chih-Hong Cheng

llms fine-tuning efficiency

arXiv Machine Learning

Jul 28

The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning

arXiv:2607. 23711v1 Announce Type: new Abstract: LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting.

By Peng Xie

llms fine-tuning

arXiv Machine Learning

Sep 2

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

arXiv:2609.00762v1 Announce Type: new Abstract: Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where...

By Wentao Ye, Zhanming Shen, Zhiqing Xiao, Yao Ding, Haobo Wang, Gang Chen

llms fine-tuning

arXiv Machine Learning

Aug 24

Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs

arXiv:2608.20873v1 Announce Type: new Abstract: Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint,...

By Zifeng Liu, Zhiyong Du, Yaxin Lu, Yiming Mao, Zhenhe Wang, Wenqi Shi, Zhengkun Jing

llms fine-tuning efficiency

arXiv AI

3d ago

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

arXiv:2605.07111v3 Announce Type: replace-cross Abstract: Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater represe...

By Haozhan Tang, Xiuqi Zhu, Xinyin Zhang, Boxun Li, Virginia Smith, Kevin Kuo

llms fine-tuning data-engineering

arXiv Machine Learning

Jul 7

SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

arXiv:2607. 04306v1 Announce Type: new Abstract: Distilling a fine-tuned teacher into a LoRA-adapted student is a standard recipe for parameter-efficient compression, but output-level KD does not explicitly control which rank-$r$ weight subspace the adapter occupies.

By Omer Tariq, Syed Muhammad Raza, Jeongbae Son

fine-tuning efficiency safety

One email a morning, machine-written

Related stories

Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration

The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation