arXiv:2601.12075v2 Announce Type: replace
Abstract: Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information...
By Mehrdad Farahani, Franziska Penzkofer, Richard Johansson
arXiv:2607. 09889v1 Announce Type: cross Abstract: Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic compute and a cache that grows with the sequence.
By Siddharth Pal, Viktoria Rojkova
The paper surveys memory‑centric compression techniques for large language models, framing long‑context handling as a memory‑management problem. It distinguishes compute‑active memory from compute‑inactive memory, and models operations such as writing, offloading, recall, and forgetting as state transitions within an information lifecycle. The authors propose a unified cost model and argue that future research should focus on dynamic, lifecycle‑aware memory management rather than static compression.
By Chaogui Gou
arXiv:2606. 26806v1 Announce Type: new Abstract: Long-running language agents need more than memory access.
By Haoliang Han
arXiv:2603. 23530v2 Announce Type: replace-cross Abstract: Large language models often fail to satisfy formatting instructions when they must simultaneously perform demanding tasks.
By Avni Mittal
arXiv:2605. 12705v2 Announce Type: replace Abstract: How can we train models whose post-trained capabilities survive subsequent fine-tuning?
By Lawrence Feng, Gaurav R. Ghosal, Jacob Mitchell Springer, Ziqian Zhong, Aditi Raghunathan
The paper introduces an action‑class diagnostic framework for multi‑turn tool‑calling in large language model agents, breaking failures into action‑class miscalibration and action‑execution failure across a four‑class action space (TOOL_CALL, ASK, REFUSE, CONFIRM). It defines a self‑revealing upper bound (Acc GAR) to expose state‑grader masking of miscalibration and shows that miscalibration is a significant, previously hidden failure mode, especially for heavily tool‑trained families. The study demonstrates that calibration can be reshaped by context‑only perturbations, but the effects vary widely across models and perturbation mechanisms, underscoring the need for diagnostics beyond aggregate accuracy.
By Kangjia Zhao, Jiajun Li, Haozhan Shen, Wei Chow, Linfeng Li, Hang Song, Lingdong Kong, Chen Zhi, Tiancheng Zhao, Songhua Liu, Jianwei Yin
arXiv:2610.22381v1 Announce Type: new
Abstract: When a fact becomes harder to elicit from a language model, has the model lost it, or has it merely become less accessible? Parameter-updating test-tim...
By Zhe Zhou
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
By Kai Ruan, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun
arXiv:2610.27724v1 Announce Type: new
Abstract: Dense linear probes assess task readability from language model (LM) representations, but do not directly reveal how many features suffice for predicti...
By Kaiyang Wan, Yuxia Wang
The paper introduces Double-Take (DT), a memory architecture designed to improve capacity, binding, and selectivity in long-context processing. DT addresses two key failures in large-chunk test-time training: interference from single-step writes and the inability of self-associative writes to form cross-position bindings, by employing multi-step unpinned updates, cross-position mixing, and bidirectional change-sensitive writes. Evaluations on genomic data show DT achieving high recall and AUROC across increasing context sizes, and zero-shot transfer from human to mouse, demonstrating the importance of these three memory requirements.
By Vakul Raghavan, Utkarsh Grover, Mingyang Mao, Chengwen Wang, Xiaomin Lin
arXiv:2607. 17545v1 Announce Type: new Abstract: Language agents depend on memory across interactions.
By Qingcan Kang, Mingyang Liu, Shixiong Kai, Kaichao Liang, Zhentao Tang, Yuqi Cui, Tao Zhong, Mingxuan Yuan