The paper investigates which linguistic operations can be replaced by latent task representations instead of in‑context prompts. A 2.6 M‑parameter network learns to encode the geometry of a few‑shot support set and applies an additive update to a frozen GPT‑2 model, achieving zero‑shot performance comparable to or better than 10‑shot in‑context learning for forward inflection and lemmatization. However, for arbitrary pairings such as antonymy, the latent approach only reaches about half the performance of in‑context learning, indicating that productive rules amortize well while memorized pairings do not.
By Gunmay Jhingran
arXiv:2610.02391v1 Announce Type: new
Abstract: When a large language model solves a mathematical problem, its reasoning is largely hierarchical, and the solution often branches at a few tokens where...
By Zeyong Zhang, Tung Sum Thomas Kwok, Tengfei Ma, Mengjia Xu
arXiv:2410.02343v2 Announce Type: replace
Abstract: Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer int...
By Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Anastasia Voznyuk, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov
arXiv:2609.13534v1 Announce Type: new
Abstract: We identify \textbf{Harmfulness Propagation Dynamics (HPD)}: for harmful prompts, the projection of the last-token hidden state onto a learned harm dir...
By Noor Islam S. Mohammad, Ulu\u{g} Bayaz{\i}t
arXiv:2609.14127v1 Announce Type: cross
Abstract: We derive exact local responses for attention interventions, allowing candidate edits to be scored from a cached baseline and one backward pass. The...
By Julie Huang, Maggie Chlon, Gregory Gutin, Leon Chlon
arXiv:2607. 04222v1 Announce Type: new Abstract: Interpretability methods aim to reveal the features represented inside large language models (LLMs).
By Amit LeVi, Elad David, Max Fomin
FishBack introduces a pullback Fisher geometry approach for activation steering in transformers, challenging the common Euclidean assumption of intermediate activation spaces. By deriving a closed‑form steering direction based on the Fisher information metric of the softmax layer, the method achieves target concept changes with minimal off‑target distortion, especially in early and middle layers. Experiments on GPT‑2 Small, Llama‑3‑8B, and Qwen3‑8B demonstrate significant reductions in off‑target KL divergence compared to existing steering baselines.
By Sihan Wang, Jiayi Zhao, Qingyan Cao, Hongbo Yao, Lin Shu
ParaBridge is a self‑distillation method that uses a temporary paralinguistic instruction scaffold during training to teach a speech language model when non‑lexical cues should influence dialogue responses. By providing dense, full‑vocabulary next‑token targets from the scaffolded view while the scaffold‑free model generates its own replies, ParaBridge stabilizes inference‑time behavior without requiring curated dialogues or external reward models. Experiments on Qwen3‑Omni show significant gains on safety and empathy benchmarks while preserving general performance across multiple tests.
By Yuxiang Wang, Qinke Ni, Shengbo Cai, Wan Lin, Liqiang Zhang, Zhizheng Wu
arXiv:2608. 10251v1 Announce Type: cross Abstract: A transformer's answer lives on one axis: the direction its unembedding reads.
By Mark Oskin
arXiv:2608. 02665v1 Announce Type: cross Abstract: A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form.
By Yongxi Zhou, Junwei Yao, Yuanzhe Liu, Zihan Dong, Wenbo Ye, Jiaxi Wen, Lai Yun Choi
arXiv:2606. 11198v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) systems inject external knowledge to improve LLM outputs, yet the format of injected content -- distinct from its semantic relevance -- can independently distort the model's attention distribution.
By Yuqi Zhang, Di Zhang
The paper evaluates how three large mixture‑of‑experts models (Alibaba, OpenAI, NVIDIA) can be fine‑tuned to reason in a low‑resource language, specifically Greek. Accuracy metrics show little change, but the authors uncover significant qualitative improvements: after supervised fine‑tuning, models reason in Greek on ~98% of items, with better grammaticality and retained general ability. Reinforcement learning with pre‑registered rewards further eliminates reasoning‑channel leaks and format skips, while the Greek‑reasoning habit remains robust to an accuracy‑only gradient.
By Ayoub Kirouane, Christos Petrocheilos