arXiv Machine Learning October 8, 2026 By Alina Sudakov, Guy Bar-Shalom, Fabrizio Frasca, Haggai Maron

Learning Cross-Model Activation Alignments with Explicit Many-to-Many Layer Maps

Read the original on arXiv Machine Learning →

The paper introduces MATCHA, a method that learns explicit many-to-many layer maps between independently trained language models. MATCHA decomposes the alignment into a layer map—an explicit target‑by‑source matrix—and a shared feature map, allowing joint learning of layer correspondence and feature transformation from prompts. Experiments on 42 model pairs across three families show that MATCHA reconstructs target activations more faithfully and improves retrieval metrics, while enabling transfer of activation‑space interventions between models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Sep 15

Ventriloquist LLMs: Linear Alignment of Late-Stage Representations

arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...

By Matt Gorbett, Suman Jana

llms rag nlp fine-tuning safety

arXiv Machine Learning

1d ago

Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment

arXiv:2604.03867v2 Announce Type: replace Abstract: Steering vectors have emerged as a lightweight and effective approach for aligning large language models (LLMs) at inference time, enabling modulat...

By Soham Gadgil, Chris Lin, Su-In Lee

llms safety

arXiv Machine Learning

Oct 3

Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models

The paper extends mechanistic interpretability of large language models by modeling concepts as low‑dimensional non‑linear manifolds rather than linear subspaces. It introduces a concept‑based alignment (CBA) score to compare these manifolds across layers and models, revealing block structures in intermediate layers, a shift from syntax‑dominated to mixed syntactic‑semantic concepts, and training‑dependent multilingual sharing. The study also shows that alignment patterns differ across model families and training stages, with adjacent stages aligning more closely than distant ones.

By Tido Specht, Elias Benedict Krey, Nils Neukirch, Nils Strodthoff

llms computer-vision reinforcement-learning fine-tuning safety

arXiv AI

Jun 18

From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging

arXiv:2506. 14126v2 Announce Type: replace-cross Abstract: Modern deep learning is increasingly characterized by the use of open-weight foundation models that can be fine-tuned on specialized datasets.

By Stefan Horoi, Guy Wolf, Eugene Belilovsky, Gintare Karolina Dziugaite

fine-tuning

arXiv Machine Learning

Jul 31

Transporting Task Vectors across Different Architectures without Training

arXiv:2602. 12952v3 Announce Type: replace Abstract: Adapting large pre-trained models to downstream tasks often produces task-specific parameter updates that are expensive to relearn for every model variant.

By Filippo Rinaldi, Aniello Panariello, Giacomo Salici, Angelo Porrello, Simone Calderara

llms

arXiv Machine Learning

Jun 11

Understanding Sample Efficiency in Predictive Coding

arXiv:2605. 11911v2 Announce Type: replace Abstract: Predictive Coding (PC) is an influential account of cortical learning.

By Gaspard Oliviers, Elene Lominadze, Rafal Bogacz

safety

One email a morning, machine-written

Related stories

Ventriloquist LLMs: Linear Alignment of Late-Stage Representations

Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment

Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models

From Memorization to Parameter Interference: How Overtraining Experts Harms Model Merging

Transporting Task Vectors across Different Architectures without Training

Understanding Sample Efficiency in Predictive Coding