arXiv Machine Learning August 13, 2026 By Chencheng Zhu

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

Read the original on arXiv Machine Learning →

arXiv:2608. 11797v1 Announce Type: new Abstract: Model merging by task arithmetic works until it doesn't, and the field diagnoses why with magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 5

Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling

arXiv:2608. 03842v1 Announce Type: cross Abstract: When a language model fails on surface-perturbed input (typos, OCR noise, homophones), "which layer is responsible" has three natural operationalizations: where representations diverge most (sensitivity), where restoring clean activations recovers the prediction (causality), and where a small adapter can repair the damage (compensatory capacity) - and we show these three layer maps dissociate.

By Nathan Labiosa, David Buff, Ena Nayak, Erica Donno

llms computer-vision

arXiv Machine Learning

Jun 5

Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models

arXiv:2606. 05378v1 Announce Type: new Abstract: We test whether a single screen-and-ablate recipe -- identify attention-head circuits by task-pattern selectivity, then verify by causal ablation against a matched-random null -- produces consistent mechanistic claims across model families.

By Yongzhong Xu

llms benchmarks

arXiv Machine Learning

Aug 11

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

arXiv:2608. 09490v1 Announce Type: new Abstract: Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function.

By Chencheng Zhu, Xiaoyang Li, Taotao Cai

llms fine-tuning safety

arXiv Machine Learning

Aug 12

Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers

arXiv:2605. 07284v2 Announce Type: replace Abstract: A late-layer change learned during post-training may work on the base model's earlier state, or it may depend on earlier computation learned with it.

By Yifan Zhou

llms fine-tuning

arXiv Machine Learning

Jun 2

Task Structure Reverses Layerwise State Encoding in Sequence Models

arXiv:2606. 00926v1 Announce Type: new Abstract: Mechanistic studies of sequence models often treat layerwise state encodings as architectural traits: recurrent models concentrate readable state, attention-based models distribute it.

By Yuhang Jiang

llms fine-tuning benchmarks

arXiv AI

1d ago

Gathered, Not Admitted: How Attention Brings a Latent Variable into Verbalizable Form

arXiv:2608. 15022v1 Announce Type: new Abstract: Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly.

By Parsa Mazaheri

llms benchmarks

Related stories

Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling

Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers

Task Structure Reverses Layerwise State Encoding in Sequence Models

Gathered, Not Admitted: How Attention Brings a Latent Variable into Verbalizable Form