What do a language model's hidden states say about the organization of a single text? From one forward pass, without training, we score every token position on two properties.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.
The paper argues that language operates with two parameters: amplitude, which measures how often words co‑occur, and phase, a signed relational factor that determines how co‑activated meanings combine and can reverse a meaning’s contribution. Unlike amplitude, phase is not captured by standard word embeddings or transformer attention weights and is indexed to individuals and dyadic interactions. The authors propose six empirical predictions to test phase’s role and suggest that future language models should incorporate agent‑indexed, phase‑bearing semantic states.
arXiv:2510. 04120v2 Announce Type: replace-cross Abstract: Large language models (LLMs) achieve strong performance on metaphor detection and interpretation tasks, yet it remains unclear what such behavioral success reveals about metaphor processing.
arXiv:2408.11827v2 Announce Type: replace Abstract: Understanding how language models compose meaning from linguistic input remains a central problem in interpretability research. Mechanistic studies...
The paper introduces Contrastive Projection, a method that reads a transformer’s internal states by differencing the hidden states of two closely matched prompts and projecting the difference through the unembedding layer. This approach cancels shared components and highlights the distinctions between prompts, effectively revealing steering vectors and domain-to-domain mappings such as metaphor. The technique is training‑free, operates at every position, sub‑layer, and head, and has been validated across multiple architectures and initialization seeds.
The study investigates how disrupting conceptual versus referential information in short narratives affects human reading and large language model (LLM) processing. In humans, conceptual disruptions cause a strong, localized processing cost that peaks early and declines quickly, while referential disruptions produce weaker, gradually decreasing effects that are more influenced by sentence boundaries. In LLMs, both disruptions appear immediately at the manipulated word; surprisal patterns mirror human reading, whereas output-layer representations show that referential disruption initially causes a larger displacement before both types decay following a power-law.
MechaTerp-TRACE is a new framework that systematically ablates individual components of language models to measure their causal contribution to producing a named entity. By applying TRACE to thirteen instruction‑tuned dense decoder models, the study finds that a small set of positionally fixed components consistently carry the most influence across models and prompts, while the remaining support is evenly distributed. This suggests that entity knowledge is largely embedded in generic generation machinery rather than in isolated, findable components.