arXiv:2607. 22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly understood beyond limited domains.
The paper investigates the intrinsic dimension (ID) of large language model (LLM) representations as an indicator of linguistic complexity. By comparing ID across model layers for coordination vs. subordination, right‑branching vs. center‑embedding, and unambiguous vs. ambiguous attachment, the authors find consistent ID differences that align with established complexity contrasts. Experiments across six LLMs, including representational similarity and layer pruning analyses, confirm that more complex phenomena produce higher ID profiles, with peaks occurring at different layers for each contrast.
arXiv:2606. 14347v1 Announce Type: new Abstract: Large language models exhibit strong multilingual capabilities, however, their internal representations are difficult to interpret.
arXiv:2510.01030v2 Announce Type: replace Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust repres...
Protein language models (PLMs) have transferred the latest advances from natural language processing to computational biology. These models, trained on large corpora of protein sequence data, are widely used to translate amino acid sequences into latent-space embeddings, ready for use in diverse downstream tasks (DTs).
arXiv:2607. 05679v1 Announce Type: cross Abstract: Language models (LMs) exhibit problematic biases, such as stereotypes.
arXiv:2606. 19857v1 Announce Type: cross Abstract: Large language models (LLMs) are commonly prompted and interfaced with human-readable natural language, even when the intended reader is another model.
The study examined how the choice of corpus affects estimates of dependency distance in language. By comparing 38 pairs of treebanks from the same language, the authors found that cross-treebank agreement was only moderate, with nearly 40% of language orderings reversed when switching treebanks. Treebank selection explained about 29% of the variance, a discrepancy that far exceeds within-treebank sampling error and persists across multiple preprocessing settings, yet all treebanks still supported the principle of dependency-length minimization.
arXiv:2609.00416v1 Announce Type: new Abstract: Probing studies have established that syntactic information is decodable in early and middle transformer layers, but what happens to that information i...
The paper studies how transformer representations evolve across layers by examining the intrinsic dimensionality (ID) of token embeddings and their neighborhood structures. It finds that closed‑class tokens expand and collapse earlier than open‑class tokens, and that these changes are linked to shifts in local geometry. The authors compare encoder and decoder models, showing distinct layer‑wise behaviors, and demonstrate that geometric features alone can predict a token’s part‑of‑speech and reveal how semantic content changes across layers.
arXiv:2603. 19066v2 Announce Type: replace-cross Abstract: Four-term word analogies (A:B::C:D) are classically modeled geometrically as parallelograms: adding the vector B-A+C produces D.
arXiv:2607. 12195v1 Announce Type: cross Abstract: Semantic memory retrieval can be conceptualized as navigation through conceptual space.