The paper investigates language‑subspace interventions designed to enhance multilingual reasoning by removing language‑related components from mid‑layer activations. Using MATH and MGSM datasets on four open‑weight LLMs, the authors examine whether these subspaces capture translation variation, whether projecting them out eliminates language information, and whether they improve accuracy without harming response language. Results show that the subspaces capture substantial translation variation but do not fully remove language signals, and that interventions can boost accuracy at the cost of language fidelity, highlighting the need to evaluate more than just accuracy.
By Shaanil Punglia, Dadhichi Telwadkar, Vaibhav Rathore, Siddharth Sureka, Neha Bhargava
ROSE is a Risk‑Aware Orthogonal Subspace Navigation framework designed for lifelong knowledge editing in multimodal large language models. It limits parameter updates to the orthogonal complement of previously edited subspaces and uses a dynamic risk mask to protect critical factual parameters, thereby reducing catastrophic forgetting. During inference, ROSE applies prototype‑anchored semantic gating and perturbation‑driven subspace fusion to activate edited knowledge precisely for relevant queries while minimizing interference on unrelated inputs.
By Lingyun Song, Ziyao Chen, Kang Pan, Xiaolin Han, Yifei Zhang, Xiaoqi Wang, Yudai Pan, Xuequn Shang
arXiv:2604.02830v3 Announce Type: replace
Abstract: Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsib...
By Yujing Wang, Yuanbang Liang, Yukun Lai, Hainan Zhang, Hanqi Yan
arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.
By Shibo Chu, Yuze Liu, Tiehua Zhang, Zhishu Shen, Lianghua He, Haofen Wang, Zhijun Ding
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
By Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He
arXiv:2502. 15543v4 Announce Type: replace-cross Abstract: Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence.
By Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong
The paper examines how new vocabulary tokens are added to language models for generative recommendation tasks. It shows that the common practice of initializing these tokens as the mean of existing embeddings collapses them into a degenerate subspace, hindering fine‑tuning. The authors propose Grounded Token Initialization (GTI), which places new tokens at semantically meaningful positions in the pretrained embedding space using linguistic supervision, and demonstrate that GTI outperforms mean initialization and other adaptation methods across several benchmarks.
By Daiwei Chen, Zhoutong Fu, Chengming Jiang, Haichao Zhang, Ran Zhou, Tan Wang, Chunnan Yao, Guoyao Li, Rui Cai, Yihan Cao, Ruijie Jiang, Fedor Borisyuk, Jianqiang Shen, Jingwei Wu, Ramya Korlakai Vinayak
arXiv:2610.41169v1 Announce Type: new
Abstract: The ability to integrate new information and deduce its consequences is central to learning, understanding, and reasoning in large language models (LLM...
By Alexander Detkov, Dashiel Karish, Matthew Oleynikov, Matt Thomson
The paper introduces DarkLens, a lightweight method for detecting faithfulness hallucinations in large language models (LLMs). It estimates context reliance by projecting hidden states onto the dark space—directions weakly aligned with the unembedding matrix—and uses the resulting Dark Matter Ratio (DMR) as a direct score of context utilization. Combined with a parametric knowledge score, DarkLens operates within a single forward pass and outperforms many baselines on QA and summarization benchmarks across several open‑weight LLMs.
By Alexandra Kuleshova, Andrey Volodichev, Daria Kotova, Alexey Zaytsev
arXiv:2608.29610v1 Announce Type: new
Abstract: The current alignment tuning paradigm for Large Language Models (LLMs) prioritizes surface-level behaviors -- fluency, safety, and tonal consistency. W...
By Chenghao Yang
arXiv:2610.20067v1 Announce Type: new
Abstract: Large Language Models (LLMs) frequently encounter conflicts between internal parametric knowledge and external contextual knowledge, yet the mechanisms...
By Yibin Lou, Yi Hu, Jianhui Chen, Chenpeng Wang, Muhan Zhang, Liangming Pan
arXiv:2606. 29164v1 Announce Type: cross Abstract: Latent reasoning models perform multi-step inference directly in hidden-state space, yet the structure of these latent reasoning trajectories remains poorly understood.
By Arun Vignesh Malarkkan, Manan Roy Choudhury, Utkarsh Byahut, Yash Ravindra Charde, Vivek Gupta, Yanjie Fu