arXiv Machine Learning
Jul 21
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models
arXiv:2607. 16741v1 Announce Type: new Abstract: B\"urger et al.
WavePhaseNet: A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)
arXiv:2602. 14419v2 Announce Type: cross Abstract: This paper reformulates Transformer/Attention mechanisms in Large Language Models (LLMs) through measure theory and frequency analysis, theoretically demonstrating that hallucination is an inevitable structural limitation.
Related stories
arXiv AI
Jun 3
Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization
arXiv:2606. 03022v1 Announce Type: cross Abstract: Hallucination in Large Language Models (LLMs), characterized by the generation of content inconsistent with contextual facts or logical constraints -- remains a persistent challenge for reliable deployment.
arXiv Computation and Language
Aug 24
Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
The paper shows that large language models (LLMs) naturally organize their hidden state manifolds into small‑world networks, enabling efficient multi‑hop reasoning. By converting similarity matrices into unweighted graphs, the authors trace connectivity between distant semantic anchors and find a sharp topological phase transition: deep reasoning layers compress conceptual distances into paths bounded by six semantic hops, while early syntactic layers remain fragmented. The framework is applied to zero‑shot hallucination detection in Retrieval‑Augmented Generation, revealing that factual generations preserve a ~3‑hop structure, whereas hallucinations collapse the topology.
arXiv AI
Sep 1
Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
arXiv:2605.01609v2 Announce Type: replace-cross Abstract: We find that transformer concept representations systematically anti-concentrate in the spectral tail of the unembedding covariance, encoding...
arXiv Computation and Language
Aug 25
Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models
arXiv:2408.11827v2 Announce Type: replace Abstract: Understanding how language models compose meaning from linguistic input remains a central problem in interpretability research. Mechanistic studies...
arXiv Machine Learning
Jul 14
One mechanism for many mental spaces: a shared router over a value slot in language models
arXiv:2607. 10248v1 Announce Type: cross Abstract: Language builds discourse contexts other than the actual: a painting, a belief, a memory, a hypothetical.
arXiv AI
Jul 14
Laguerre Geometry for Interpreting Large Language Models
arXiv:2607. 10578v1 Announce Type: new Abstract: Existing hypotheses represent a concept in an LLM as a single point, a linear direction, or a Gaussian cluster, yet it remains unclear how and why such structures emerge.
arXiv Computation and Language
Aug 25
DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning
DynHD is a method for detecting hallucinations in diffusion large language models (D‑LLMs) by focusing on token‑level uncertainty and its evolution during the denoising process. It introduces a semantic‑aware evidence construction module that filters out non‑informative structural tokens and highlights uncertainty in informative tokens, and a reference evidence generator that models the expected trajectory of uncertainty, enabling a deviation‑based detector to identify hallucinations. Experiments show DynHD outperforms existing baselines while being more efficient across various benchmarks and backbone models.
arXiv Machine Learning
Jun 19
Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models
arXiv:2606. 19404v1 Announce Type: new Abstract: Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality.
arXiv AI
Jul 28
D-Score: A Spectral Hidden-State Signal for Hallucination Detection in Large Language Models
arXiv:2607. 24586v1 Announce Type: cross Abstract: Large Language Models can produce fluent text that is false, unsupported by the available evidence, or inconsistent with information that appears to be internally represented by the model.
arXiv AI
4d ago
Coherence-Aware Distributional Evaluation of Open-Ended Text Generation
arXiv:2609.34240v2 Announce Type: replace-cross Abstract: Existing open-ended generation metrics measure likelihood, lexical diversity, or distributional similarity in generic representation space, y...
arXiv AI
Sep 7
Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models
The article presents a technical manual for an open toolkit designed to measure how transformer language models individuate word meanings across different contexts. It introduces the concept of a "bridge form"—a single word that appears unchanged in multiple domains but with distinct senses—and outlines a full pipeline from specifying these forms to extracting layer-wise representations, computing silhouette-based separation metrics, and visualizing results. The manual details each design choice and its intended methodological safeguards, emphasizing that it serves as a methodological reference rather than reporting empirical findings.