The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper investigates how semantic meaning is arranged locally within large language models. By using two controlled probes—one cross‑lingual and one refinement probe—on four decoder‑only LLMs, it finds that after removing dominant variance directions, sentence pairs rank by semantic relatedness, semantic factors cluster around an invariant core with structured variation, refinement tightens neighborhoods, and sentence‑level versus token‑local readouts reveal complementary geometric patterns. These results suggest that LLM semantics form local anisotropic neighborhoods that contract when meaning is refined.
arXiv:2608.29034v1 Announce Type: cross Abstract: A wide range of methods have been proposed for interpreting language models, delivering important insights into their inner workings. However, differ...
arXiv:2610.30934v1 Announce Type: new Abstract: Activation verbalizers translate the internal representations of large language models into human-readable text. The recently proposed natural language...
Activation verbalizers translate the internal representations of large language models into human-readable text. The recently proposed natural language autoencoders (NLAs) learn this by reconstructing...
arXiv:2609.01356v1 Announce Type: new Abstract: Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they tra...
MechaTerp-TRACE is a new framework that systematically ablates individual components of language models to measure their causal contribution to producing a named entity. By applying TRACE to thirteen instruction‑tuned dense decoder models, the study finds that a small set of positionally fixed components consistently carry the most influence across models and prompts, while the remaining support is evenly distributed. This suggests that entity knowledge is largely embedded in generic generation machinery rather than in isolated, findable components.