arXiv Machine Learning

In-Context Binding Capacity in Language Models

arXiv:2609. 30634v1 Announce Type: new Abstract: How many assignments can a language model recall before it loses track of which value belongs to which entity?

arXiv AI
Jul 14

Remembering Distinct Items, Not Tokens: A Learnable Dirichlet-Process Cache Between State-Space Models and Attention

arXiv:2607. 09889v1 Announce Type: cross Abstract: Fixed-state sequence models compress an unbounded past into a bounded state, which caps their associative recall at roughly the state dimension; attention escapes the cap by keeping a key-value entry for every token, at quadratic compute and a cache that grows with the sequence.

By Siddharth Pal, Viktoria Rojkova
arXiv Computation and Language
Oct 16

From Long Context to Selective Memory:A Survey of Memory-Centric Compression for Large Language Models

The paper surveys memory‑centric compression techniques for large language models, framing long‑context handling as a memory‑management problem. It distinguishes compute‑active memory from compute‑inactive memory, and models operations such as writing, offloading, recall, and forgetting as state transitions within an information lifecycle. The authors propose a unified cost model and argue that future research should focus on dynamic, lifecycle‑aware memory management rather than static compression.

By Chaogui Gou
arXiv AI
Sep 2

Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling

The paper introduces an action‑class diagnostic framework for multi‑turn tool‑calling in large language model agents, breaking failures into action‑class miscalibration and action‑execution failure across a four‑class action space (TOOL_CALL, ASK, REFUSE, CONFIRM). It defines a self‑revealing upper bound (Acc GAR) to expose state‑grader masking of miscalibration and shows that miscalibration is a significant, previously hidden failure mode, especially for heavily tool‑trained families. The study demonstrates that calibration can be reshaped by context‑only perturbations, but the effects vary widely across models and perturbation mechanisms, underscoring the need for diagnostics beyond aggregate accuracy.

By Kangjia Zhao, Jiajun Li, Haozhan Shen, Wei Chow, Linfeng Li, Hang Song, Lingdong Kong, Chen Zhi, Tiancheng Zhao, Songhua Liu, Jianwei Yin
arXiv Machine Learning
Oct 12

Double-Take: Improving Capacity, Binding, and Selectivity In Long-Context Within Bidirectional Memory

The paper introduces Double-Take (DT), a memory architecture designed to improve capacity, binding, and selectivity in long-context processing. DT addresses two key failures in large-chunk test-time training: interference from single-step writes and the inability of self-associative writes to form cross-position bindings, by employing multi-step unpinned updates, cross-position mixing, and bidirectional change-sensitive writes. Evaluations on genomic data show DT achieving high recall and AUROC across increasing context sizes, and zero-shot transfer from human to mouse, demonstrating the importance of these three memory requirements.

By Vakul Raghavan, Utkarsh Grover, Mingyang Mao, Chengwen Wang, Xiaomin Lin