Large language models (LLMs) exhibit a human‑like working‑memory pattern on the N‑back task, with performance dropping as memory load increases and errors mainly on lure items. Two hypotheses—precision limits of softmax attention and interference from competing items—were tested across 15 pretrained LLMs and minimal transformers. Results show that interference, not precision, constrains LLMs’ working‑memory performance, as suppressing non‑target item identities improves sensitivity and larger models benefit more from reducing interference.
By Dongyu Gong
The paper introduces a three-level evaluation framework—behavioral deployment, LM-head readout, and probe recoverability—to distinguish whether a language model fails a syntactic test by not encoding structure or by failing to use it. Using a trilingual control-dependency benchmark, the authors find that probe recoverability consistently exceeds LM-head readout, which in turn exceeds behavioral deployment across seven models and three languages, with the largest gap observed in Qwen3-0.6B Instruct. Layer-localized activation patching shows that instruction tuning shifts the decoded layer later, suggesting decoding favors surface shortcuts and that behavioral evaluation understates what models encode while probing alone overstates what they deploy.
By Zhenyan Lu, He Wang, Xiaohui Huang
The paper investigates when a language model actually learns task information versus when its readout (unembedding matrix) learns to express that information. Using a new parameter‑trajectory cross‑coding method, the authors find that hidden states can encode task data early in training, but the readout reorganizes only in a narrow window (0.4–1.1% of Pythia’s pretraining). By varying the readout learning rate, they show that faster readout learning reduces the gap between hidden‑state encoding and output expression without affecting final performance.
By Matteo He, William F. Shen, Alex Iacob, Andrej Jovanovic, Xinchi Qiu, Nicholas D. Lane
arXiv:2409. 02228v2 Announce Type: replace Abstract: When language models (LMs) are trained to forget (or "unlearn'') a skill, how precisely does their behavior change?
By Eric Zhang, Leshem Choshen, Jacob Andreas
arXiv:2609.16229v1 Announce Type: new
Abstract: Machine unlearning aims to remove specific knowledge from a trained large language model (LLM) without retraining from scratch. Existing methods modify...
By Pingzhi Li, Jinhao Duan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Sijia Liu, Tianlong Chen
The paper introduces Update‑Matched Unlearning (UMU), a method that applies the same simulated relearning update to both a defended model and an undefended reference to better preserve plasticity while resisting relearning of deleted knowledge. Experiments on Llama and Qwen models with TOFU and FaithUn datasets show that UMU reduces harmful and benign relearning by 30.4 and 9.9 points on average and increases new‑task accuracy by 3.5 points, supporting the reference‑timing hypothesis. The study demonstrates that robust unlearning should selectively prevent recovery of deleted knowledge without suppressing the model’s ability to learn new tasks.
By Jie Li, Youming Tao, Bowei He, Meng Ding
arXiv:2608. 05224v1 Announce Type: new Abstract: Large language models fine-tuned on human behavioural data have emerged as general-purpose cognitive proxies, but the scale this requires, and whether these models process task structure or exploit statistical shortcuts, remain open questions.
By Nick Oh, Fernand Gobet
The paper investigates why large vision‑language models sometimes misclassify harmful memes, attributing failures to either missing internal evidence or poor routing of evidence to the output. Using sparse autoencoders, role‑conditioned probes, and causal interventions on Gemma‑3 and Qwen3.5, the authors show that sparse readouts consistently outperform native predictions across six harmful content benchmarks, revealing a readout gap that is largely due to routing rather than representation. The study also demonstrates that calibration‑only routing recovers most of the performance gap and that the issue persists across languages and is not solely driven by OCR signals.
By Girish A. Koushik, Diptesh Kanojia, Helen Treharne
The study compares two ways of obtaining predictions from language models fine‑tuned on customer behavior: scoring answer tokens directly versus generating a written rationale and then scoring the resulting answer. Across 13 model‑domain cells covering four retail tasks, scored readouts consistently rank outcomes more accurately than generated readouts, with an AUC improvement ranging from 1.5 to 14.5 points. The authors also find that a third readout—eliciting a probability before any verdict—improves calibration but only when outcome rates are represented in training, and they recommend using generated rationales for interpretability while relying on scored heads for ranking.
By Touchapon Kraisingkorn, Krittin Pachtrachai, Wachiravit Modecrua
arXiv:2609.00746v1 Announce Type: new
Abstract: Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone's text capability, with the damage concentrated on tasks that re...
By Minsik Choi, Geewook Kim, Young Geun Kim
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu
arXiv:2610.15235v1 Announce Type: new
Abstract: Despite remarkable progress in language understanding and generation, decoder language models continue to exhibit striking failures on numerical data....
By Weimin Lyu, Jingfen Zhang, Lingdao Sha, Peng sun, Xiaowei Chen, Vincent Gao, Rui Song