The paper investigates where and how large language models encode signals that indicate answer correctness. By examining hidden states, token probabilities, residual-stream features, attention, and their combinations, the authors find that correctness signals are concentrated in the answer span and that different signal families complement each other. Fusing these signals improves robustness, especially under distribution shifts, and can be used to control retrieval in downstream tasks.
By Yuansen Liu, Yixuan Tang, Anthony Kum Hoe Tung
arXiv:2606. 01923v1 Announce Type: cross Abstract: Large Language Models (LLMs) frequently exhibit "contextual disregard" when faced with input evidence that conflicts with their internal parametric memory, leading to persistent factual hallucinations.
By Mingkuan Zhao, Yide Gao, Wentao Hu, Suquan Chen, Tianchen Huang, Zhenhua An, Zetao Chang, Xiayu Sun, Yuheng Min
arXiv:2607. 02460v1 Announce Type: cross Abstract: Post-training large language models (LLMs) without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain.
By Zhuowei Chen, Xiang Lorraine Li
The paper investigates how different prompt adaptation techniques—discrete prompt optimization, prompt tuning, and prefix tuning—affect the internal processing of dense and mixture-of-experts language models. It finds that the adaptation location determines how task information reaches the output: text prompts mainly influence response construction, input-level prompts modify content representations, and layer-wise prefixes exert direct control via attention. In MoE models, these differences also lead to distinct expert‑routing patterns, while generic representation‑level metrics fail to reliably distinguish effective adaptations from placebo effects.
By Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov, Andrew Tsukanov, Timofey Karyagin, Alexander Miasnikov, Gleb Molodtsov, Valentin Samokhin, Aleksandr Beznosikov
arXiv:2608. 16697v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogeneous architectures.
By Aniri, Chen Yilin, Jinhe Bi, Junfei Guo, Donglai Ran, Xu Bian, Zengjie Jin, Yujun Wang, Yijun Tian, Volker Tresp, Fei Shen, Tat-Seng Chua, Yunpu Ma
The paper investigates how attention dynamics evolve across recurrent depth in language models, finding that attention support stabilizes early while hidden states and outputs take longer. It proposes WISE, a training‑free method that uses full attention in early steps and then reuses the discovered sparse working set for later steps, preserving performance on multi‑hop QA tasks. Experiments show that WISE maintains quality up to 2K context, offers measurable speedups, and highlights the importance of recurrent discovery of attention support.
By Ke Wan, Chen Chen