UpgradeBench is a decision‑centric longitudinal benchmark that evaluates how fine‑tuned language‑model specialists should be handled when new base‑model releases occur. It covers four consecutive Qwen releases, a continuation checkpoint, six tasks, two model sizes, and OLMo checkpoints with known training lineage, and examines whether retraining, adapter transfer, or other recovery strategies improve specialist performance. The benchmark reveals that upgrade gains vary by task and release interval, that direct adapter copying is sensitive to pretraining distance, and that teacher relabeling can recover specialists without new annotations.
"whyItMatters":"The study provides actionable insights into the cost‑effective management of specialist models across model releases, showing how to balance retraining effort with performance gains."
By Ye Chen, Weining Zhang
The paper investigates how a higher‑level controller in a hierarchical latent reasoner decides whether to preserve or replace a lower‑level strategy. Experiments across different training seeds show that learned revision timing can produce varied policies, but none surpasses a fixed, forced timing policy on the same checkpoint. The study highlights that state‑dependent control can alter actions without improving task performance, and that timing decisions are order‑sensitive and largely captured by a strong fixed schedule.
By Ayushi Chadha
arXiv:2609.00874v1 Announce Type: new
Abstract: As agentic systems become compound systems, increasingly important decisions move above task execution itself: when should a higher-level controller pr...
By Ayushi Chadha
The paper investigates why constrained updates in supervised fine‑tuning (SFT) still lead to forgetting by developing a functional‑path analysis that decomposes changes in old loss into controlled direction, gradient staleness, probe mismatch, and finite‑step effects. It introduces a conditional refresh budget and identifies a restoring direction provided by informative replay, showing that the placement of constraints relative to Adam and the granularity of constraints significantly affect retention across code‑label audits. The study also explores checkpointed generation, replay schedules, and sequential‑task comparisons, demonstrating that subspace regularization, projection, and supervised replay differentially influence backward transfer and content retention.
By Qingsen Ma, Dianyun Wang, Jiahong Liu, Irwin King
arXiv:2610.15882v1 Announce Type: new
Abstract: A stored fact may stop being valid when a user changes a preference or an event replaces an earlier one. We study whether a language model can learn th...
By Sam Siavoshian, Omar Ramadan, Amir K. Saeed, Benjamin A. Johnson, Benjamin M. Rodriguez
arXiv:2609. 04875v1 Announce Type: cross Abstract: Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache.
By Chao Yao, Yangbo Wei, Zhen Huang, Junhong Qian, Chenle Chen, Shaoqiang Lu, Chen Wu, Lei He
The paper argues that evaluating continual knowledge‑updating methods solely at a final checkpoint and a single adapter rank can be misleading. By fixing a periodic hierarchy and comparing it to cumulative replay on a 24‑month Wikidata stream, the authors show that the apparent best method changes depending on the evaluation month, replay LoRA rank, and query formulation. They recommend reporting performance trajectories and capacity sweeps, and only declaring a robust winner when the ranking remains stable across the evaluation region.
By Heejin Choi
The paper introduces Update‑Matched Unlearning (UMU), a method that applies the same simulated relearning update to both a defended model and an undefended reference to better preserve plasticity while resisting relearning of deleted knowledge. Experiments on Llama and Qwen models with TOFU and FaithUn datasets show that UMU reduces harmful and benign relearning by 30.4 and 9.9 points on average and increases new‑task accuracy by 3.5 points, supporting the reference‑timing hypothesis. The study demonstrates that robust unlearning should selectively prevent recovery of deleted knowledge without suppressing the model’s ability to learn new tasks.
By Jie Li, Youming Tao, Bowei He, Meng Ding
MemoProAgent tackles memory‑driven proactive assistance by mining long‑horizon personal memories to decide when an assistant should intervene. The authors introduce MemoProBench, a benchmark pairing egocentric video with year‑long, distractor‑rich user memories and human‑verified trigger windows, and show that existing models perform poorly (≤15%). MemoProAgent uses a graph‑organized memory store and a lightweight RL stage to improve accuracy from <10% to 42.4% without training.
By Shenglai Zeng, Xiao Yang, Tiffany Ma, Xinyuan Zhang, Nanshu Wang, Deren Lei, Kai Sun, Xilun Chen, Tammy Stark, Xianhui Zhu, Yuetai Li, Rulin Shao, Tianxin Wei, Rakesh Wanga, Anuj Kumar, Jiliang Tang, Wen-tau Yih, Xin Luna Dong
The paper introduces ProPerMem, a benchmark for memory-driven proactive assistance that pairs egocentric video with long, distractor-rich personal memories and human-verified trigger windows. Baseline models perform poorly, achieving at most 15% accuracy, while an oracle-driven GPT-5.3-Chat model reaches 96% when given evidence inside the trigger window. Building on these insights, the authors develop ProPerAgent, a plug‑and‑play system that retrieves relevant memory from a graph store and uses lightweight reinforcement learning to improve proactive intervention accuracy to 42.4% without training.
By Shenglai Zeng, Xiao Yang, Tiffany Ma, Xinyuan Zhang, Nanshu Wang, Deren Lei, Kai Sun, Xilun Chen, Tammy Stark, Xianhui Zhu, Yuetai Li, Rulin Shao, Tianxin Wei, Rakesh Wanga, Anuj Kumar, Jiliang Tang, Wen-tau Yih, Xin Luna Dong
arXiv:2610.22381v1 Announce Type: new
Abstract: When a fact becomes harder to elicit from a language model, has the model lost it, or has it merely become less accessible? Parameter-updating test-tim...
By Zhe Zhou
The paper investigates when it is better to return an existing draft answer or revise it using retrieved evidence in retrieval‑augmented QA systems. By grading both the draft and its candidate revision with the same correctness judge, the authors define a paired effect called recoverability and train policies to predict it before revision. Experiments on 25,870 open‑domain questions show that a recoverability‑based scorer outperforms a draft‑correctness scorer across multiple Llama setups, improving accuracy–revision trade‑offs and closing a significant portion of the oracle gap, though it still applies harmful revisions in a substantial fraction of cases.
By Nicholas Kashani Motlagh, Tim Anderson, Jeremy Gwinnup, Grant Erdmann