arXiv Computation and Language October 23, 2026 By Andrea Blasi N\'u\~nez, Gianluca Barmina, Federico Torrielli, Thiusius Rajeeth Savarimuthu, Peter Schneider-Kamp, Lukas Galke Poech

Verbalizing Activation Differences with Natural Language Autoencoders

Read the original on arXiv Computation and Language →

The paper investigates natural language autoencoders (NLAs) as tools for verbalizing activation differences in large language models. It examines whether NLAs capture compositional structure, generalize to fine‑tuned models, recover fine‑tuning objectives, and can be specialized for model diffing. Experiments across four models show that NLAs largely succeed in these tasks, though they do not yet match state‑of‑the‑art diffing methods on emergent misalignment detection.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Hugging Face Trending Papers

2d ago

Verbalizing Activation Differences with Natural Language Autoencoders

Activation verbalizers translate the internal representations of large language models into human-readable text. The recently proposed natural language autoencoders (NLAs) learn this by reconstructing...

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 7

The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models

arXiv:2604. 19139v3 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) continue to evolve through alignment techniques such as Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI, a growing and increasingly conspicuous phenomenon has emerged: the proliferation of verbal tics--repetitive, formulaic linguistic patterns that pervade model outputs.

By Shuai Wu, Xue Li, Yanna Feng, Yufang Li, Zhijun Wang, Ran Wang

llms reinforcement-learning benchmarks safety

arXiv Computation and Language

2d ago

Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training

arXiv:2610.29843v1 Announce Type: new Abstract: Large language models are typically trained with next-token prediction (NTP), which rewards predicting specific tokens rather than explicitly accountin...

By Laya Iyer, Chen Shani

llms nlp

arXiv Machine Learning

3d ago

On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination

arXiv:2610.18727v2 Announce Type: replace Abstract: Natural Language Autoencoders (NLAs) are proposed as oversight tools for language models: an Activation Verbaliser (AV) writes an English explanati...

By Kevin Yandoka Denamgana\"i

llms multimodal

Hugging Face Trending Papers

3d ago

Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training

Large language models are typically trained with next-token prediction (NTP), which rewards predicting specific tokens rather than explicitly accounting for the semantic similarities between different...

arXiv Machine Learning

Jun 16

Emergent retokenization symmetry in large language models: phenomenology and applications

arXiv:2606. 15521v1 Announce Type: cross Abstract: Tokenization introduces representational redundancy: under a fixed token vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string.

By Kanishk Jain, Matthew Day, Tankut Can

llms computer-vision nlp benchmarks

One email a morning, machine-written

Related stories

Verbalizing Activation Differences with Natural Language Autoencoders

The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models

Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training

On the Monitorability of Natural Language Autoencoders: Identification vs Discrimination

Every Part Has Its Role: Part-of-Speech Effects in LLM Post-Training

Emergent retokenization symmetry in large language models: phenomenology and applications