The paper investigates where fine‑tuned behaviors in language models are instantiated by patching the key‑value (KV) cache between a LoRA‑fine‑tuned model and its base version. Results show that certain behaviors, such as German responses or emergent misalignment, can be transferred or removed by swapping KV cache segments, and that the impact grows with the amount patched and is concentrated at a few pivotal positions like system‑prompt and chat‑template tokens.
By Jonas Loos, Oliver Eberle
arXiv:2606. 28358v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) aims to enhance the trustworthiness of Large Language Models (LLMs) by grounding their outputs in external documents, often using inline citations for verifiability.
By Ian van Dort (University of Amsterdam), Maria Heuss (University of Amsterdam)
The paper investigates whether language models truly reveal their internal decision process when faced with conflicting evidence. In a conflicting‑context question‑answering task, the authors find that models can silently commit to one source, a hidden source commitment that can be detected via a residual source‑choice direction. They show that steering this direction can change the chosen source, while conflict disclosure behaves differently and is only partially steerable.
By Samuel Dawit Assefa, Jae Won Cho
arXiv:2603.13911v2 Announce Type: replace
Abstract: A language model can give the wrong answer even when the correct answer is decodable from its intermediate states. To study this gap between decoda...
By Valeria Ruscio, Keiran Thompson
arXiv:2607. 04572v2 Announce Type: replace Abstract: Large language model (LLM) tutors may have access to teacher notes, answer keys, rubrics, or retrieved solutions while producing student-facing explanations.
By Bonan Shen, Dingyan Shang, Youting Wang, Tao Ning, Bowen Liu
The paper investigates how entity‑replacement responses can be reused across different English factual questions when model weights are fixed. By injecting hidden‑state changes from two source questions into a target question about the same entity, the authors compare predicted log‑probability shifts with actual measurements, finding that hidden reuse can reduce no‑change error by up to 49% on certain models and that a later protocol can achieve 58‑68% success on three cohorts. The study also notes that larger checkpoints, chat formats, and new entities affect replication, and that relation‑neutral changes at entity tokens lower relation‑selective error more effectively than random changes.
By Jinkun Chen, Xuemin Yu, Sijia Han, Vlado Keselj
The paper investigates how a language model’s first answer to a yes/no question influences its subsequent answer to a different question about the same entity, a phenomenon termed self‑commitment. By fixing the initial answer and measuring changes in the second across 3,600 question pairs for 22 open‑weight models (0.5B–72B parameters), the authors find that each answer writes a commitment direction into the residual stream, and the similarity of these directions predicts answer coupling. They show that ablating a shared direction reduces coupling significantly, while the reading circuit for this direction varies with model size, instruction tuning, and prompts, and that sparing the attention sink during ablation preserves low perplexity cost in most models.
By Marcus Armstrong
arXiv:2610.14958v1 Announce Type: new
Abstract: Activation steering has recently been used as a memory for skills: past experiences of an agent are stored as steering vectors and retrieved later. Fac...
By Vedant Jumle
The paper investigates how large language models (LLMs) such as Qwen, Llama, and Gemma use internal knowledge when answering questions. By performing layer‑wise interventions on the hidden state after the question, the authors compare how different request directions (pair‑conditioned vs. global) and answer types (noun, adjective, code) influence the model’s routing of information. The study finds that the influence of request direction varies across models and layers, with some models showing a sustained routing effect while others do not, highlighting distinct patterns of early readability, causal steering, and later content dependence.
By Wenkang Wei, Yuan Fang, Renhe Jiang, Hong Cheng, Xingtong Yu
arXiv:2610.32191v1 Announce Type: new
Abstract: A collection date records when a source revision was acquired, not when its contents changed. Can assigning this metadata redirect temporal question an...
By Minsu Oh, Seunghui Jwa, Soo Kyun Kim, Yeo-Chan Yoon
arXiv:2610.32055v1 Announce Type: new
Abstract: Large language models (LLMs) often exhibit sycophancy, altering their answers to align with user suggestions even on objective factual and reasoning ta...
By Anxin Guo, Chirag Pabbaraju, Sara Ahmadian, Kamesh Munagala, Aravindan Vijayaraghavan
arXiv:2610.35694v1 Announce Type: new
Abstract: Conversational QA systems should fix errors when users push back and signal which answers to trust. We split each follow-up into its speech act (re-ask...
By Duc-Vinh Tran