The paper studies structural priming in language model production by conducting controlled sentence‑completion experiments on dative constructions. Results show that language models exhibit priming effects, especially when sentences are semantically coherent, with stronger relative increases for double‑object datives and larger absolute increases for prepositional‑object datives. The study also finds that primed completions involve more lexico‑semantic repetition, indicating that priming operates across syntactic, lexical, and semantic levels.
By Giulia Pucci, Ruizhe Li, Arabella Sinclair
The paper investigates how language models avoid overgeneralizations by distinguishing between two types of indirect negative evidence: preemption and entrenchment. Through controlled rearing experiments on models trained on child‑caregiver conversations, the authors find that models do not exhibit verb‑specific preemption but show weak abstract preemption. Analysis of training dynamics suggests that competing structures act as indirect positive evidence rather than negative in the verb‑specific condition.
By Yixuan Wang, Freda Shi, Kanishka Misra
arXiv:2608. 04021v1 Announce Type: cross Abstract: Cloze-style probes that vary how often a target token appears implicitly assume that more copies of a target affect prediction the same way regardless of where the readout slot sits.
By Han-yu Wang
The study investigates whether large language models retain traces of abandoned syntactic analyses after encountering garden‑path sentences. Across 24 models ranging from 124 M to 70 B parameters, the authors find that a residue of the initial, incorrect analysis influences later object attachment, while an adaptation toward the resolved structure scales with disambiguation error. Activation‑patching experiments show that the residue is carried by noun‑phrase states computed under the object analysis, and that overriding these states largely removes the effect.
By Xiang Ding
arXiv:2610. 11765v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention.
By Dianqiao Lei, Kevin Qinghong Lin, Pan Lu, Philip Torr, James Zou
The paper studies how large language models (LLMs) handle the discourse particles "just" and "only" in natural conversation. Behavioral experiments show that both humans and LLMs prefer these particles and are more sensitive to "just" than "only", while instruction tuning enhances this sensitivity. Mechanistic analysis identifies specific units in the models that respond to each particle, revealing an asymmetric overlap in their internal representations.
By William Sheffield, Jiahui Chen, Amy Zhang, Alex Warstadt, Junyi Jessy Li, Kanishka Misra
arXiv:2608. 14681v1 Announce Type: cross Abstract: Words recur constantly in natural language use, yet it remains unclear whether language models reactivate prior representations or re-evaluate repeated words afresh, and whether post-training changes this default behavior.
By Jinglei Ren, Yuyue Wang
arXiv:2608. 12387v1 Announce Type: cross Abstract: Positional biases such as recency and primacy effects have been documented in large language models (LLMs), yet the underlying mechanism by which these models make their evaluations remains poorly understood.
By Jasin Cekinmez, Addison J. Wu, Thomas L. Griffiths
The paper investigates whether language models can identify sentences from their training data by using exact duplication counts from publicly released corpora for two model families, OLMo‑2 and Pythia. It finds that for typical duplication levels, models show only a weak trace of exposure, with a rank correlation near –0.08, and that strong signals only appear when a sentence appears roughly a thousand times, at which point fame rather than memory dominates. The study also demonstrates that common membership tests can be misleading, as changing a single word does not alter the model’s preference, and that controlling for register can significantly improve detector performance.
By Arman Nik Khah
The paper introduces Controlled Evidence Aggregation (CEA), a method that manipulates the counts and order of two symbols in fixed prompts to probe a frozen Transformer’s preference between two answer tokens. Using Balanced Utility Causal Selection (BUC), the authors identify small sets of attention heads whose ablation removes a large portion of the count effect, and show that these same heads also influence performance on a natural language task. The study demonstrates that symbolic probes can uncover compact head sets with causal impact on both synthetic and real-world language decisions.
By Jipeng Li, Pramod Khargonekar, Yanning Shen
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
arXiv:2606. 00935v1 Announce Type: new Abstract: We test whether a relational-style intervention delivered during functional collapse in a small language model produces post-collapse behavior distinguishable from technical feedback, from a lexically-matched scrambled control, and from each of the two pragmatic dimensions in isolation.
By Franco Santana, Horacio Vico