arXiv:2610.18674v1 Announce Type: cross
Abstract: Large language models rely on prompt context to perform in-context learning, follow formats, and adapt to local evidence. This ability, however, also...
By Jinghan Zheng, Jiajun Shi
arXiv:2610.32309v1 Announce Type: new
Abstract: Why can a task-preserving rewrite turn a correct answer into a failure while a larger internal change leaves the answer intact? We develop a direction-...
By Zhibo Hu, Chen Wang, Yanfeng Shu, Liming Dong, Hye-young Paik, Liming Zhu
arXiv:2610.27580v1 Announce Type: cross
Abstract: Factual recall is a lookup: some representation inside the network serves as the key, and what that key is made of decides which queries can retrieve...
By Yuchen Guo, Zhuoyuan Mao, Yiwen Guo
The paper demonstrates that language models can acquire new capabilities from post‑training data even when the training text is unrelated to the target task. Using a method called Active Taskless Distillation (ATD), the authors show that a single word from a teacher model can transfer knowledge to a student model without any target‑task examples or teacher logits. Experiments on Qwen2.5-1.5B reveal significant performance gains on HumanEval+ and improvements in scientific knowledge, commonsense reasoning, and reading comprehension across various model families.
By Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li, Haofan Wang, Yichen Gong
The study investigates how task context reshapes the roles of shared attention heads in large language models. By applying identical graph facts and measuring interventions on attention heads and downstream activations, the authors show that task changes redistribute influence across five open‑weight models, causing the same heads to respond to different facts and affect later components differently. Linear response maps before and after activation edits reveal significant reductions in prediction error, indicating that task context alters both the facts processed and the downstream impact of shared components.
By Sovesh Mohapatra, Dani S Bassett
The paper investigates which linguistic operations can be replaced by latent task representations instead of in‑context prompts. A 2.6 M‑parameter network learns to encode the geometry of a few‑shot support set and applies an additive update to a frozen GPT‑2 model, achieving zero‑shot performance comparable to or better than 10‑shot in‑context learning for forward inflection and lemmatization. However, for arbitrary pairings such as antonymy, the latent approach only reaches about half the performance of in‑context learning, indicating that productive rules amortize well while memorized pairings do not.
By Gunmay Jhingran