arXiv AI
A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 7
InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
arXiv:2506. 07406v3 Announce Type: replace-cross Abstract: Understanding the internal representations of large language models (LLMs) is a central challenge in interpretability research.
arXiv Computation and Language
Aug 25
Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models
arXiv:2408.11827v2 Announce Type: replace Abstract: Understanding how language models compose meaning from linguistic input remains a central problem in interpretability research. Mechanistic studies...
Hugging Face Trending Papers
4d ago
Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability
Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is unde...
arXiv Machine Learning
Jun 18
From Mechanistic to Compositional Interpretability
arXiv:2605. 08934v2 Announce Type: replace Abstract: Mechanistic interpretability aims to explain neural model behaviour by reverse-engineering learned computational structure into human-understandable components.
arXiv AI
3d ago
Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability
This survey reviews tensor methods applied to large language models, framing them through a seven‑stage lifecycle (tokenization, embeddings, pre‑training, adaptation, compression, inference, interpretability) and a component view (embeddings, attention, feed‑forward networks). It offers unified notation, theoretical foundations, and comparative analyses of tensorization strategies for Transformer components, while highlighting evaluation protocol differences and model scale effects. The paper also introduces a new metric, ρ_gap, to quantify the gap between theoretical memory savings and actual system‑level speedup, and connects tensor techniques to related efficiency and probabilistic methods.
arXiv AI
Aug 7
Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors
arXiv:2608. 06300v1 Announce Type: new Abstract: Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first language (L1) or age.