arXiv Machine Learning

Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning

arXiv:2606. 07560v1 Announce Type: cross Abstract: Function-vector (FV) heads (Todd et al.

arXiv Machine Learning

6d ago

Orientation, not magnitude: the causal structure of task-vector interference in merged language models

arXiv:2608. 11797v1 Announce Type: new Abstract: Model merging by task arithmetic works until it doesn't, and the field diagnoses why with magnitudes: layerwise representation bias, deviations from cross-task linearity, parameter overlap.

By Chencheng Zhu

llms efficiency safety

arXiv AI

Jun 16

Not All Skills Help: Measuring and Repairing Agent Knowledge

arXiv:2606. 15390v1 Announce Type: cross Abstract: LLM agents can improve without weight updates by accumulating natural-language skills from experience, but current systems entrust every decision about which skills to keep and how to apply them to LLM judgment alone.

By Yixuan Wang, Yiyang Zhou, Yiming Liang, Congyu Zhang, Fuxiao Liu, Jiawei Zhou, Huaxiu Yao

llms agents benchmarks

arXiv Machine Learning

Jun 24

One Ruler: A Same-Hands Re-Evaluation of Bivariate Causal Direction on Tuebingen, with a Parameter-Free Compression Baseline

arXiv:2606. 23767v1 Announce Type: new Abstract: Headline accuracies on the Tuebingen cause-effect pairs are routinely compared across papers even though each is measured under its authors' own protocol -- different pair subsets, weightings, model-selection, and decision rates.

By Wietse Stienstra

efficiency

arXiv Machine Learning

Jul 21

When Can You Debias an LLM Judge? Identifiability Limits, a Test, and Designs for Top-k Ranking

arXiv:2607. 02104v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as cheap, scalable judges that compare candidate outputs pairwise.

By Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

llms safety

arXiv Machine Learning

Jun 9

Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning Updates

arXiv:2606. 07596v1 Announce Type: new Abstract: Fine-tuning often introduces spurious correlations alongside task knowledge, causing systematic failures on underrepresented groups.

By Edward Sun, Dmitrii Troitskii

fine-tuning benchmarks

arXiv Machine Learning

Aug 4

Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation

arXiv:2608. 01676v1 Announce Type: cross Abstract: Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific content on model output.

By Xingyu Ren, Youran Sun, Chugang Yi, Haizhao Yang

efficiency benchmarks

arXiv Machine Learning

Aug 11

When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

arXiv:2608. 09490v1 Announce Type: new Abstract: Task arithmetic treats fine-tuning displacements as composable directions in weight space, yet it remains unclear when parameter addition reflects predictable changes in model function.

By Chencheng Zhu, Xiaoyang Li, Taotao Cai

llms fine-tuning safety

arXiv AI

2d ago

Split the Labor: Separating Evidence Interpretation from Decision Aggregation

arXiv:2608. 14509v1 Announce Type: new Abstract: Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt.

By Zhelun Wu

llms

arXiv AI

Aug 11

Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

arXiv:2608. 08829v1 Announce Type: cross Abstract: Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task.

By Muhammad Faishal Adly Nelwan, Alfan Farizki Wicaksono

llms rag safety

arXiv Machine Learning

Jul 30

Learning What Matters: Supervising Global Context Pruning with Causal Evidence Sets

arXiv:2607. 21692v2 Announce Type: replace Abstract: Sparse attention prunes a long context to the blocks a model needs, and the usual selector is distilled from a dense teacher's attention.

By Jim Allchin

efficiency

arXiv Machine Learning

Aug 3

ALIVE: Warnings Before Exclusion in Budgeted Multi-Source Learning

arXiv:2607. 29400v1 Announce Type: new Abstract: A routing decision can be revised at the next transaction, but a latched source exclusion persists across later decisions.

By Xiyang Zhang, Hongzhi Wang, Yuanhe Tian

arXiv AI

Jul 7

Transplanting, inverting, and preventing a misalignment persona: method-conditional emergent misalignment in Qwen2.5

arXiv:2607. 04510v1 Announce Type: cross Abstract: Emergent misalignment (EM) -- the broad misbehaviour a language model acquires after fine-tuning on narrow harmful data -- is mediated in Qwen2.

By Lyndon Drake (University of Oxford), Zandi Eberstadt (University of Oxford)

llms fine-tuning

Related stories