arXiv Machine Learning October 3, 2026 By Yimiao Yu, Florentin Guth

Localizing Transfer Between Memorization Tasks

Read the original on arXiv Machine Learning →

The paper investigates how pre‑training on random input‑output mappings (memorization tasks) can transfer to downstream tasks. It discovers two unexpected patterns: equivalent transfer, where each pre‑training epoch saves roughly one fine‑tuning epoch, and non‑equivalent transfer, where pre‑training on a mismatched task can be more efficient than training directly on the downstream task. Ablation studies reveal that transfer consists of a trivial magnitude‑driven effect in the last layer and a non‑trivial structure‑driven effect linked to covariance in other layers.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Sep 1

Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models

The study investigates how post‑training of large autoregressive language models (ARMs) into masked diffusion models (MDMs) affects their internal computation. Across two 7B ARM‑MDM families and four diagnostic tasks, the authors find that MDMs retain much of the ARM’s high‑attribution pathways on prefix‑dominant tasks, but reorganize computation toward earlier layers on globally constrained tasks. Component‑level probes reveal that ARMs depend on sharply specialized components, whereas MDMs show weaker specialization and more diffuse output‑space alignment.

By Injin Kong, Hyoungjoon Lee, Yohan Jo

llms diffusion safety

arXiv Machine Learning

4d ago

Early Learning Shapes Later Directions Of Representation Change In Continual Learning

arXiv:2609.36081v1 Announce Type: new Abstract: Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure th...

By Yuantao Deng, Jinnuo Liu, Kaizhen Tan, Yuchen Liu

arXiv Machine Learning

Sep 2

Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training

arXiv:2609.01170v1 Announce Type: new Abstract: Large language models exhibit a modular internal organization that mirrors well-studied functional networks of the human brain, but how this organizati...

By Guangqi Li, Yongxin Li

llms

Hugging Face Trending Papers

5d ago

Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases

The paper introduces ATLAS, a method that uses retained-domain activation atlases to guide task-specific fine‑tuning of language models. By providing local reference centers and directional filters, ATLAS learns a low‑rank residual that adapts to new tasks while preserving existing behavior. Experiments on Qwen3‑8B and other backbones show lower retained‑output KL divergence, fewer rewritten answers, and more stable responses compared to seven baselines.

llms fine-tuning

arXiv AI

Sep 21

Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models

Fine‑tuning reshapes internal representations of large language models, affecting attention patterns and layer‑wise activations. The study shows that components identified by EAP as important for task performance cluster in specific layers, yet these layers do not align with those undergoing the largest representational changes. Additionally, overlapping EAP components across different tasks do not guarantee cross‑task transfer and can even degrade performance when tasks differ in nature.

By Lingfang Li, Procheta Sen, Shubham Das, Danushka Bollegala

llms fine-tuning

arXiv Statistics ML

2d ago

In-context Learning of Single-index Targets: Comparing Kernel and Feature Learners

arXiv:2610.01712v1 Announce Type: cross Abstract: In-context learning (ICL) enables a pretrained model to infer a task from demonstrations without updating its parameters. While much of the existing...

By Haotian Gu, Yizhou Xu, Lenka Zdeborov\'a