arXiv Computation and Language

Calibrated concept profiles: making the reading of a model's neurons a measurement

The paper introduces calibrated concept profiles, a method that treats direct neuron readings as measurable quantities. By sampling prompts from a defined distribution and generating two independent batteries, the authors compute a per‑neuron profile expressed in the neuron’s own activation spread, complete with noise units and a minimum resolvable change. The approach is applied to OpenCoder‑1.5B across checkpoints spanning two trillion training tokens, providing unit‑scaled, chance‑level‑aware, and checkpoint‑specific construct attribution for each neuron.

arXiv Computation and Language
Nov 9

Representational drift in a code model: behaviour settles, which neurons carry it does not

The study investigates representational drift in the OpenCoder‑1.5B language model by tracking how neurons encode 169 Python constructs during pretraining on two trillion tokens. While the loss for 16 grammar‑fixing constructs quickly reaches a floor, the specific neurons that carry these constructs continue to change throughout training, with no stable carrier set emerging even at the final checkpoint. The authors find that removing the identified carriers incurs a measurable performance penalty, yet the carrier sets remain dynamic, challenging the hypothesis that they settle over time.

By Piotr Wilam
arXiv AI
Oct 16

Learning to Read Out: Unembedding Dynamics in Language Model Pretraining

The paper investigates when a language model actually learns task information versus when its readout (unembedding matrix) learns to express that information. Using a new parameter‑trajectory cross‑coding method, the authors find that hidden states can encode task data early in training, but the readout reorganizes only in a narrow window (0.4–1.1% of Pythia’s pretraining). By varying the readout learning rate, they show that faster readout learning reduces the gap between hidden‑state encoding and output expression without affecting final performance.

By Matteo He, William F. Shen, Alex Iacob, Andrej Jovanovic, Xinchi Qiu, Nicholas D. Lane
arXiv AI
Oct 28

Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data

The paper demonstrates that Mixture-of-Experts (MoE) models can avoid the need for calibration data when selecting which expert channels to load during decoding. Instead of using a calibrated threshold, a top‑k selection strategy—choosing the k highest‑scoring channels per token—achieves comparable or better performance without calibration. Additionally, the authors propose a reader that uses a ternary sketch and low‑rank mapping to recover skipped channels, achieving 98.1% of full‑model accuracy while reading only 17.8% of expert weights and speeding up inference by up to 2.69× compared to optimized vLLM.

By Geunhyoung Kim, Sung Ju Hwang
arXiv Machine Learning
Aug 27

Amplifying, Not Learning: The Price of Out-of-Distribution Generalization in AI-Text Detection

The paper shows that AI‑text detectors, rather than learning a clear AI‑versus‑human boundary, amplify an inherited predictability axis that already exists in language models. This amplification causes detectors to over‑flag fluent, formal human writing while missing high‑temperature AI outputs, and the bias persists across languages, code, and detector architectures. A training‑free operator can relocate the bias but cannot erase it, underscoring that the unfairness is a structural cost of out‑of‑distribution generalization.

By Alexander Smirnov
Hugging Face Trending Papers
Oct 27

Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data

The paper demonstrates that Mixture-of-Experts (MoE) decoding can avoid the need for calibration data by using a top‑k channel selector instead of a calibrated threshold. Top‑k selection consistently matches or outperforms threshold‑based methods across various read budgets and tasks, eliminating issues like repetition‑loop collapse. Additionally, the authors propose a reader that uses a ternary sketch and low‑rank mapping to recover skipped channels, achieving 98.1% of full‑model accuracy while reading only 17.8% of routed‑expert weights and speeding up MoE blocks by up to 2.69× compared to optimized vLLM.

arXiv Computation and Language
Sep 10

Through the Looking Glass: Directly Reading and Writing Transformers

The paper investigates how many transformer components influence a token prediction by measuring the absolute contribution of each unit and channel to the logit. It finds that thousands of components contribute to a single prediction, yet a small subset—often just dozens—carries the majority of the predictive mass. Across models ranging from 124 M to 7 B parameters, the proportion of the model involved in a prediction remains around one to three percent, independent of size, and the study demonstrates that specific components can be directly read and written to modify model behavior without additional training.

By Mark Oskin
arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga
arXiv Machine Learning
Aug 31

How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution

The paper introduces Concept-Targeted Attribution (CTA), a method that trains attribution graphs to explain the emergence of internal concept representations in language models, rather than just the final token prediction. CTA produces probe-specific circuits that reveal which internal computations drive a linear probe’s accuracy, and cross-layer transcoders demonstrate that these graphs contain predictive structure across multiple concept categories. Causal ablations show that probe-targeted and logit-targeted graphs capture distinct mechanisms, with probe-relevant features affecting internal concept scores and logit-relevant features altering generated tokens.

By Vedant Palit, Florent Draye, Terry Jingchen Zhang, Bernhard Sch\"olkopf, Zhijing Jin