arXiv Machine Learning

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

arXiv:2608. 09490v1 Announce Type: new Abstract: Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function.

arXiv AI
Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko
arXiv Machine Learning
Jul 28

Wrong Design Intent Is Worse Than None: A Derangement-Control Diagnosis of Header Conditioning in CAD Program Completion

arXiv:2607. 23191v1 Announce Type: new Abstract: Fine-tuned code LLMs can be conditioned on a lightweight design-intent header to steer parametric CAD generation, but whether the model actually reads the header's content has not been tested under a metric independent of the conditioning itself, nor with a causal control.

By Yang Xiao