arXiv Machine Learning

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

arXiv:2608. 11797v1 Announce Type: new Abstract: Model merging by task arithmetic works until it doesn't, and the field diagnoses why with magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

arXiv Machine Learning

Aug 5

Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling

arXiv:2608. 03842v1 Announce Type: cross Abstract: When a language model fails on surface-perturbed input (typos, OCR noise, homophones), "which layer is responsible" has three natural operationalizations: where representations diverge most (sensitivity), where restoring clean activations recovers the prediction (causality), and where a small adapter can repair the damage (compensatory capacity) - and we show these three layer maps dissociate.

By Nathan Labiosa, David Buff, Ena Nayak, Erica Donno

llms computer-vision

arXiv Machine Learning

Jun 5

Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models

arXiv:2606. 05378v1 Announce Type: new Abstract: We test whether a single screen-and-ablate recipe -- identify attention-head circuits by task-pattern selectivity, then verify by causal ablation against a matched-random null -- produces consistent mechanistic claims across model families.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Aug 11

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

arXiv:2608. 09490v1 Announce Type: new Abstract: Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function.

By Chencheng Zhu, Xiaoyang Li, Taotao Cai

llms fine-tuning safety

arXiv Machine Learning

Aug 12

Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers

arXiv:2605. 07284v2 Announce Type: replace Abstract: A late-layer change learned during post-training may work on the base model's earlier state, or it may depend on earlier computation learned with it.

By Yifan Zhou

llms fine-tuning

arXiv Machine Learning

Jun 2

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606. 00926v1 Announce Type: new Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, attention-based models distribute it.

By Yuhang Jiang

llms fine-tuning benchmarks

arXiv AI

1d ago

Gathered, Not Admitted: How Attention Brings a Latent Variable into Verbalizable Form

arXiv:2608. 15022v1 Announce Type: new Abstract: Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly.

By Parsa Mazaheri

llms benchmarks

arXiv AI

Jul 7

They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It

arXiv:2607. 03598v1 Announce Type: cross Abstract: When a person shares something with a language model, the model often answers the surface of the message rather than what the sender was doing by sending it: share a finished project and it critiques the code; share a raw late-night line and it runs a wellness check.

By Alex Kwon

llms safety

arXiv AI

Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Machine Learning

Aug 6

Wrong Design Intent Can Be Worse Than None: A Derangement-Control Diagnosis of Header Conditioning in CAD Program Completion

arXiv:2607. 23191v2 Announce Type: replace Abstract: Fine-tuned code LLMs are often conditioned on a design-intent header to steer parametric CAD generation, but whether the model reads that header's content has been tested neither under execution-level scoring nor with a causal control.

By Yang Xiao

llms fine-tuning safety

arXiv Machine Learning

Jun 9

Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning

arXiv:2606. 07560v1 Announce Type: cross Abstract: Function-vector (FV) heads (Todd et al.

By Han-yu Wang

llms

arXiv AI

Jun 8

Endogenous Resistance to Activation Steering in Language Models

arXiv:2602. 06941v2 Announce Type: replace-cross Abstract: Large language models can recover mid-generation from task-misaligned activation steering, producing explicit verbal restarts (e.

By Alex McKenzie, Keenan Pepper, Stijn Servaes, Martin Leitgab, Murat Cubuktepe, Mike Vaiana, Diogo de Lucena, Judd Rosenblatt, Michael S. A. Graziano

llms robotics fine-tuning safety

arXiv Machine Learning

Jul 28

Wrong Design Intent Is Worse Than None: A Derangement-Control Diagnosis of Header Conditioning in CAD Program Completion

arXiv:2607. 23191v1 Announce Type: new Abstract: Fine-tuned code LLMs can be conditioned on a lightweight design-intent header to steer parametric CAD generation, but whether the model actually reads the header's content has not been tested under a metric independent of the conditioning itself, nor with a causal control.

By Yang Xiao

llms fine-tuning

Related stories