arXiv AI July 9, 2026 By Varun Reddy Nalagatla

The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions

arXiv:2606. 12629v3 Announce Type: replace-cross Abstract: The standard basis of transformer hidden states is a training-free, architecture-general feature basis for detecting concepts and, in language models, steering them; with no learned dictionary.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

llms

arXiv AI

Jun 12

Bag of Dims: Training-Free Mechanistic Interpretability via Dimension-Level Sign Patterns

arXiv:2606. 12629v1 Announce Type: cross Abstract: We show that the standard basis of transformer hidden states already provides a training-free, architecture-general feature basis.

By Varun Reddy Nalagatla

llms efficiency safety

arXiv Computer Vision

Sep 4

SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval

SignSeek is a new method for learning transferable sign representations that enables efficient retrieval of signs from dictionaries using only a query video. It employs contrastive learning with saliency‑guided articulator masking, aligning same‑gloss signs across signers while focusing on the single most critical articulator per sign. Trained on 266K samples from multiple sign languages, SignSeek achieves state‑of‑the‑art cross‑corpus retrieval performance and zero‑shot generalisation to unseen British Sign Language, also improving isolated sign recognition and subtitle alignment.

By Sobhan Asasi, Ozge Mercanoglu Sincan, Richard Bowden

rag diffusion fine-tuning benchmarks safety

arXiv Machine Learning

Jul 1

Signed-Permutation Coordinate Transport for RMSNorm Transformers

arXiv:2606. 31963v1 Announce Type: new Abstract: Modern LLM workflows move coordinate-indexed objects across checkpoints: steering vectors, sparse autoencoders, top-$k$ neuron sets, attribution lists, and merge alignments.

By John Sweeney

llms fine-tuning safety

arXiv Machine Learning

Jul 21

The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models

arXiv:2607. 16741v1 Announce Type: new Abstract: B\"urger et al.

By Francesco Karim Vicidomini

llms

arXiv Machine Learning

Aug 4

Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

arXiv:2605. 28149v2 Announce Type: replace Abstract: Sparse Autoencoders (SAEs) extract interpretable features from Large Language Model activations, but standard variants enforce non-negative latents, so a bidirectional semantic axis (e.

By Bartosz Wieciech, Zmnako Awrahman, Marcin Czelej, Victor Hugo Jaramillo Velasquez, Wioletta Stobieniecka

llms efficiency safety

arXiv Machine Learning

Aug 12

Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So

arXiv:2608. 10251v1 Announce Type: cross Abstract: A transformer's answer lives on one axis: the direction its unembedding reads.

By Mark Oskin

llms benchmarks

One email a morning, machine-written

Related stories

Bag of Dims: Training-Free Mechanistic Interpretability via Dimension-Level Sign Patterns

SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval

Signed-Permutation Coordinate Transport for RMSNorm Transformers

The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models

Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations

Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So