arXiv:2606. 05079v1 Announce Type: cross Abstract: Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models (LLMs).
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2609.28117v1 Announce Type: cross Abstract: In this paper, we introduce a gradient-based head attribution strategy where the Token-level Max-Margin loss is backpropagated to the attention maps....
arXiv:2607. 13425v1 Announce Type: cross Abstract: Learning effectively from limited data is critical in domains like security where labeled examples are scarce.
arXiv:2609.13141v1 Announce Type: new Abstract: Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by selecting a small set of context u...
arXiv:2606. 29844v1 Announce Type: cross Abstract: The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particularly in long-context scenarios.
The paper investigates how the dimension of attention heads influences long‑context retrieval in language models. Experiments show that while pretraining loss and short‑context tasks are largely unaffected, wider heads significantly improve retrieval performance at fixed model sizes. Ablation studies indicate that this benefit persists even when attention weights are frozen, suggesting that wider heads enhance adaptation of other model components such as embeddings and MLPs.
arXiv:2602.01654v2 Announce Type: replace Abstract: Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing...