arXiv Machine Learning

Omega-S: A Functional Resilience Index for LLM Fine-Tuning

arXiv:2608. 03887v1 Announce Type: new Abstract: Fine-tuning a large language model on new data degrades what it previously learned.

arXiv Machine Learning

Jun 9

Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning Updates

arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.

By Edward Sun, Dmitrii Troitskii

fine-tuning benchmarks

arXiv AI

Jun 29

When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study on Tabular Data Finds the Warm-Start Is a Default Configuration, Not the Model

arXiv:2606. 21641v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been proposed as hyperparameter-optimization (HPO) advisors that "warm-start" search from prior knowledge, proposing strong configurations in very few evaluations.

By Carson Rodrigues, Oysturn Vas, Isaiah Abner DCosta, Nithish Kumar Prabhakaran

llms benchmarks

arXiv Machine Learning

23h ago

OraclePhys: A Systematic Framework for LLM Fine-Tuning on Structural Mechanics

arXiv:2608. 17162v1 Announce Type: new Abstract: What a language model internalizes from fine-tuning is usually diagnosed after the fact.

By Mingyu Li, Guorui Song, Jing Lin, Haoqian Wang

llms fine-tuning benchmarks

arXiv AI

Jul 21

TopoTuner: Topological Finetuning of Large Language Models

arXiv:2607. 16637v1 Announce Type: new Abstract: Full fine-tuning remains a strong way to adapt pretrained LLMs, but it updates all weights and can be expensive.

By Abdulkadir Erol, Yash Mahajan, Vepaul Hariprashad, Baha Rababah, Santu Karmaker, Cuneyt G. Akcora, Mubarak Shah

llms nlp fine-tuning

arXiv Machine Learning

Jul 24

Emergent Misalignment Recruits a Pre-existing Persona Subspace

arXiv:2607. 21356v1 Announce Type: new Abstract: Fine-tuning an aligned language model on a narrow stream of bad advice can make it broadly misaligned on questions unrelated to the training data, a phenomenon called emergent misalignment.

By Mohammed Suhail B Nadaf

llms fine-tuning

arXiv AI

Jun 16

Fine-Tuning a 7B Advisor on Free-Tier GPUs: An Adapter-Handoff Recipe and a Synthetic-Data Reliability Caution

arXiv:2504. 15610v4 Announce Type: replace Abstract: Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on.

By Md Millat Hosen

llms fine-tuning data-engineering

arXiv Machine Learning

Jul 20

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.

By Shahrzad Esmat, Dhawal Shah, Ali Jannesari

llms efficiency benchmarks

arXiv Machine Learning

Aug 4

Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models

arXiv:2608. 01624v1 Announce Type: cross Abstract: Adapting a language model to a task no longer requires training all of its weights, and a line of parameter-efficient methods has driven the trainable count from billions down to a handful of scalars.

By Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim, Unggi Lee

llms benchmarks

arXiv Machine Learning

Aug 5

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

arXiv:2608. 02947v1 Announce Type: new Abstract: The attention score with rotary position embeddings (RoPE) decomposes exactly into a sum over its 2D-rotation frequency pairs, and each pair's wavelength limits how far it can discriminate position.

By Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino, Takahiro Katagiri

llms rag efficiency benchmarks

arXiv AI

Jun 8

The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning

arXiv:2606. 06920v1 Announce Type: cross Abstract: Deploying Small Language Models (SLMs) on edge devices requires efficient fine-tuning strategies that adapt models to new tasks without degrading their general capabilities.

By Rahul Nair, Chun Tao

llms fine-tuning benchmarks

arXiv AI

Jun 30

Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation

arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.

By Gias Uddin, Sanjeepan Sivapiran

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 10

ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning

arXiv:2607. 07719v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning adapts a large language model to one task cheaply, but across a task sequence LoRA-style methods keep stacking low-rank updates on the same frozen weight, so each new task tends to overwrite the previous ones.

By Wentao Lu

llms fine-tuning

Related stories