arXiv AI
Ventriloquist LLMs: Linear Alignment of Late-Stage Representations
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Aug 18
Architecture-Dependent Causal Transfer of Activation States Across Large Language Models
arXiv:2608. 16347v1 Announce Type: cross Abstract: Direct communication between AI systems relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost, and latency.
arXiv Machine Learning
Jun 2
Reconsidering Positional Supervision in Masked Diffusion Language Model Training
arXiv:2601. 22947v2 Announce Type: replace-cross Abstract: Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models.
arXiv Machine Learning
Sep 11
Perturbation: A simple and efficient adversarial tracer for representation learning in language models
The paper introduces Perturbation, a method that treats representations in language models as learning conduits rather than activation patterns. By fine‑tuning a model on a single adversarial example and observing how this perturbation spreads to other inputs, the approach avoids geometric assumptions and does not identify representations in untrained models. In trained models, Perturbation uncovers structured transfer across multiple linguistic scales, indicating that language models generalize along representational lines and acquire linguistic abstractions through experience.
arXiv Machine Learning
Jun 26
The Geometry of Updates: Fisher Alignment at Vocabulary Scale
arXiv:2606. 27242v1 Announce Type: new Abstract: Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets.
arXiv Machine Learning
Oct 8
Learning Cross-Model Activation Alignments with Explicit Many-to-Many Layer Maps
The paper introduces MATCHA, a method that learns explicit many-to-many layer maps between independently trained language models. MATCHA decomposes the alignment into a layer map—an explicit target‑by‑source matrix—and a shared feature map, allowing joint learning of layer correspondence and feature transformation from prompts. Experiments on 42 model pairs across three families show that MATCHA reconstructs target activations more faithfully and improves retrieval metrics, while enabling transfer of activation‑space interventions between models.
arXiv Computation and Language
Oct 27
Five Architectures, One Shared Space Cross-Model Representational Alignment Without Weight Merging
arXiv:2610.33495v1 Announce Type: new Abstract: Combining language models is usually done by averaging weights, which works only when the models share a common starting point. We ask whether five fro...