arXiv:2508. 12448v2 Announce Type: replace-cross Abstract: In-context learning (ICL) lets large language models (LLMs) solve new tasks from prompts alone, across an ever-widening range of domains, yet the mechanisms underlying this ability remain poorly understood.
By Yeongwoo Song, Jaeyong Bae, Dong-Kyum Kim, Hawoong Jeong
arXiv:2607. 12279v1 Announce Type: cross Abstract: Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table.
By Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen
The paper introduces adVersarial Parameter Decomposition (VPD), a method that splits a language model’s parameters into simple subcomponents, each responsible for a small part of the model’s algorithm. VPD optimizes these decompositions so that the model’s input‑output behavior remains intact even when many subcomponents are removed, including adversarially chosen ones. This approach enables a new form of circuit analysis by tracing how sequences of parameter subcomponents interact to produce outputs for specific inputs.
By Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin, Bart Bussmann, Nathan Hu, Michael Ivanitskiy, Linda Linsefors, Lee Sharkey
arXiv:2608. 25897v1 Announce Type: new Abstract: Explaining deep learning models operating on time series data is crucial in various applications that require transparent and interpretable insights into model behavior.
By Xu Zheng, Zichuan Liu, Zhuomin Chen, Mayur Akewar, Janki Bhimani, Jason Liu, Mo Sha, Jingchao Ni, Wei Cheng, Dongsheng Luo
arXiv:2601. 21996v2 Announce Type: replace-cross Abstract: While Mechanistic Interpretability has identified interpretable circuits in LLMs, their causal origins in training data remain elusive.
By Jianhui Chen, Yuzhang Luo, Liangming Pan
Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires tracking how many tokens remain before a target.