The paper investigates language‑subspace interventions designed to enhance multilingual reasoning by removing language‑related components from mid‑layer activations. Using MATH and MGSM datasets on four open‑weight LLMs, the authors examine whether these subspaces capture translation variation, whether projecting them out eliminates language information, and whether they improve accuracy without harming response language. Results show that the subspaces capture substantial translation variation but do not fully remove language signals, and that interventions can boost accuracy at the cost of language fidelity, highlighting the need to evaluate more than just accuracy.
By Shaanil Punglia, Dadhichi Telwadkar, Vaibhav Rathore, Siddharth Sureka, Neha Bhargava
ROSE is a Risk‑Aware Orthogonal Subspace Navigation framework designed for lifelong knowledge editing in multimodal large language models. It limits parameter updates to the orthogonal complement of previously edited subspaces and uses a dynamic risk mask to protect critical factual parameters, thereby reducing catastrophic forgetting. During inference, ROSE applies prototype‑anchored semantic gating and perturbation‑driven subspace fusion to activate edited knowledge precisely for relevant queries while minimizing interference on unrelated inputs.
By Lingyun Song, Ziyao Chen, Kang Pan, Xiaolin Han, Yifei Zhang, Xiaoqi Wang, Yudai Pan, Xuequn Shang
arXiv:2604.02830v3 Announce Type: replace
Abstract: Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsib...
By Yujing Wang, Yuanbang Liang, Yukun Lai, Hainan Zhang, Hanqi Yan
arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.
By Shibo Chu, Yuze Liu, Tiehua Zhang, Zhishu Shen, Lianghua He, Haofen Wang, Zhijun Ding
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.
By Xiaoang Xu, Siyuan Liu, Shuo Wang, Junlan Feng, Fanyu Meng, Zhu Zhang, Jixun Wang, Xiaorong Wang, Zihan Zhou, Xin Li, Chaojun Xiao, Yiming Zhang, Huijia Wu, Liuyu Xiang, Peipei Li, Zhaofeng He
arXiv:2502. 15543v4 Announce Type: replace-cross Abstract: Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence.
By Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong