arXiv Machine Learning By Nika Zahedi, Fangyi Zhu, Anna Wu

MicroLens: Read, Trace, and Rewire the Residual Stream with Two Lenses

Read the original on arXiv Machine Learning →

MicroLens introduces an interactive viewer that overlays two decoding lenses—logit and Jacobian—onto the residual stream of a language model, allowing users to read, trace, and edit token representations on a layers‑by‑tokens grid. Experiments on Gemma‑3‑4B‑it show that the logit lens directs token outputs while the Jacobian lens influences how later tokens interpret information, enabling targeted edits. The tool reveals that key information (e.g., state or country names) can be read and manipulated early in the stream, and that correct answers often appear in the residual stream before reasoning is generated, with the Jacobian lens detecting them slightly earlier than the logit lens.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Oct 16

Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics

The paper introduces Lexical Reference Frames (LexFrame) as a method to interpret language model residual streams by treating each hidden state as having a native reference frame induced by subsequent layers. Unlike fixed lenses or point‑cloud approaches, LexFrame reads a state through the model’s remaining layers using a set number of attention rounds, achieving high coherence (0.97) and accurate state reconstruction (within 0.16 nats). The approach also provides faithful attribution, token‑level steering, and potential early‑exit savings, making every hidden state readable, attributable, and steerable.

By Jian Gu, Hongyu Zhang, Chunyang Chen, Aldeida Aleti