arXiv AI July 21, 2026 By Irina Piontkovskaia, Sergey Nikolenko

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI

Jun 10

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

arXiv:2606. 10929v1 Announce Type: cross Abstract: Task vectors, LoRA, activation steering, and random search around pretrained weights all suggest that learned behaviour can be controlled by linear directions.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Machine Learning

Aug 11

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

arXiv:2608. 09490v1 Announce Type: new Abstract: Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function.

By Chencheng Zhu, Xiaoyang Li, Taotao Cai

llms fine-tuning safety

arXiv AI

Jul 15

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter

arXiv:2607. 10203v2 Announce Type: replace-cross Abstract: Adaptive-compute world models -- early-exit or mixture-of-depths predictors that spend variable depth per step -- assume depth buys better predictions and can be routed adaptively.

By Achyuthan Sivasankar

benchmarks

arXiv AI

Jun 9

Phantom transitions in language model fine-tuning

arXiv:2606. 07559v1 Announce Type: cross Abstract: Fine-tuning a language model on contexts whose correct completion has a near-synonym competitor often fails silently.

By Vaibhav Prakash, Jayasri Dontabhaktuni

llms rag fine-tuning

arXiv Machine Learning

Aug 4

When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer

arXiv:2608. 01130v1 Announce Type: new Abstract: A broad range of models face the mismatch where they are updated through trajectory losses but are evaluated by downstream task reward.

By Yuyang Shen

llms

arXiv Machine Learning

Aug 4

Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models

arXiv:2608. 01624v1 Announce Type: cross Abstract: Adapting a language model to a task no longer requires training all of its weights, and a line of parameter-efficient methods has driven the trainable count from billions down to a handful of scalars.

By Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim, Unggi Lee

llms benchmarks

Related stories

Recoverable but Not Stationary:Local Linear Structures in Weights and Activations

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter

Phantom transitions in language model fine-tuning

When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer

Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models