arXiv:2609.39702v1 Announce Type: new
Abstract: Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choic...
By Nima H. Siboni, Vahid Rostami
arXiv:2608. 13069v1 Announce Type: new Abstract: Large language models (LLMs) are predominantly aligned to function as passive, sycophantic assistants.
By Lucia Mal\'i\v{c}kov\'a
Large language models (LLMs) are predominantly aligned to function as passive, sycophantic assistants. We challenge this default paradigm by empirically evaluating the cognitive plasticity of open-weight architectures when subjected to rigorous behavioral reprogramming.
The study investigates how layer‑wise intervention responses in language models change over the course of pretraining, using single‑block identity bypass across multiple checkpoints and model‑domain combinations. It finds that while depth ordering of responses persists, their magnitudes shift, with nearby checkpoints showing stronger rank correspondence than distant ones and large changes occurring at positions that recur across samples and transfer across evaluation domains. Controlled experiments reveal that these longitudinal changes cannot be explained by a single downstream sensitivity and depend on perturbation strength and direction, indicating that layer sensitivity is structured but dynamic.
By Shengye Tao, Yinzhu Cheng, Haihua Xie
arXiv:2606. 05378v1 Announce Type: new Abstract: We test whether a single screen-and-ablate recipe -- identify attention-head circuits by task-pattern selectivity, then verify by causal ablation against a matched-random null -- produces consistent mechanistic claims across model families.
By Yongzhong Xu
The study investigates how post‑training of large autoregressive language models (ARMs) into masked diffusion models (MDMs) affects their internal computation. Across two 7B ARM‑MDM families and four diagnostic tasks, the authors find that MDMs retain much of the ARM’s high‑attribution pathways on prefix‑dominant tasks, but reorganize computation toward earlier layers on globally constrained tasks. Component‑level probes reveal that ARMs depend on sharply specialized components, whereas MDMs show weaker specialization and more diffuse output‑space alignment.
By Injin Kong, Hyoungjoon Lee, Yohan Jo