The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2607. 22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.
The paper investigates the intrinsic dimension (ID) of large language model (LLM) representations as an indicator of linguistic complexity. By comparing ID across model layers for coordination vs. subordination, right‑branching vs. center‑embedding, and unambiguous vs. ambiguous attachment, the authors find consistent ID differences that align with established complexity contrasts. Experiments across six LLMs, including representational similarity and layer pruning analyses, confirm that more complex phenomena produce higher ID profiles, with peaks occurring at different layers for each contrast.
arXiv:2610.14476v1 Announce Type: cross Abstract: Comparing large language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity m...
The paper extends mechanistic interpretability of large language models by modeling concepts as low‑dimensional non‑linear manifolds rather than linear subspaces. It introduces a concept‑based alignment (CBA) score to compare these manifolds across layers and models, revealing block structures in intermediate layers, a shift from syntax‑dominated to mixed syntactic‑semantic concepts, and training‑dependent multilingual sharing. The study also shows that alignment patterns differ across model families and training stages, with adjacent stages aligning more closely than distant ones.
The paper introduces a framework to assess how linearly encoded linguistic relations are within language model embeddings. Using a constrained linear approximation on related and unrelated word pairs, the authors evaluate an extended BATS dataset across inflectional, derivational, lexicographic, and encyclopedic relations in GloVe, RoBERTa, and ModernBERT. Results show near-perfect linear encoding for inflectional and derivational relations, higher errors for lexicographic and encyclopedic relations—especially in one-to-many and many-to-many cases—and that RoBERTa and ModernBERT encode relations more linearly than GloVe.
Despite the success of neural models in natural language processing, their black-box nature limits interpretability and conceals the linguistic phenomena underlying their predictions. We present SLITE...