Depth pruning removes entire Transformer blocks to shrink and speed up language models, but the resulting damage is uneven across capabilities. In a study of four 1.2–1.7 B instruction‑tuned models, function calling and mathematical reasoning collapsed in every case, while other skills varied in order of degradation. Fine‑tuning a short ‘healing’ run can restore some abilities, with the choice of which blocks to retrain and the allocation of a limited token budget influencing recovery outcomes.
By Piyush Dhoka, Vedant Pandhare, Varun Inamdar, Aadarsh Pathre, Pavitha Nooji
The study compares two delivery architectures for a multi‑turn agent that uses a tool to retrieve structured records. In the SYNTHESIS architecture, all records are fed into the model’s context, leading to a dramatic drop in accuracy as the table grows beyond about 74 rows. The BYPASS architecture keeps records in session state and delivers them via a typed side channel, maintaining high accuracy (92.3%) regardless of table size across four large language models.
By Tung Thanh Hoang
arXiv:2605.19309v4 Announce Type: replace
Abstract: Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answeri...
By Yue Chen, Yihao Wang, Ziyi Tang, Yongsen Zheng, Keze Wang
arXiv:2605.26037v2 Announce Type: replace
Abstract: We test the standard RLVR tool-use recipe -- GRPO on Qwen2.5-7B-Instruct -- on a deliberately minimal knowledge-graph tool API: four Freebase navig...
By Tianda Sun, Dimitar Kazakov
WeVisDoc is a two‑stage data‑centric framework designed to improve end‑to‑end document parsing. Stage I expands coverage by adding heterogeneous data and applying structure‑preserving degradation synthesis, while Stage II evaluates residual errors with a held‑out probe and uses those diagnostics to target data construction and token budget reallocation. The resulting WeVisDoc‑4B model achieves an overall score of 95.38 on OmniDocBench v1.6 and outperforms competing parsers across all evaluated settings, with Stage II delivering notable gains on degraded tracks.
By Hao Yu, Kang Liu, Linnan Zhao, Jiabo Zhan, Chong Sun, Chen Li, Jing Lyu
The study examines how context projection—replacing older tool observations with concise, addressable excerpts—affects performance in ReVerPi, a Pi extension that archives observations and matches full to projected continuations. Across 86 source‑reading runs and 641 model requests, 15 completed pairs achieved identical success rates (12/15 per arm), while 12 boundary runs halted when the first arm failed, revealing that projection can reduce logical tokens by 25% but increase median pair tokens by 29% and total suffix requests from 35 to 55. The analysis highlights the importance of retaining all intervention boundaries, executing both arms independently, and reporting completion, token usage, and interaction details to accurately assess stopping rules and resource aggregation.
By Guangzhe Zhang
arXiv:2607. 16019v1 Announce Type: new Abstract: AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations.
By Zhaoyang Jiang, Zhizhong Fu, Zicheng Li, Yunsoo Kim, Jiacong Mi, Xuanqi Peng, Fei Teng, Honghan Wu
arXiv:2607. 13124v1 Announce Type: cross Abstract: Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires.
By Qingyu Zhang, Qianhao Yuan, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun, Xiang Li, Ming Xu, Jiarui Li, Xiuyin Zhao
arXiv:2608. 00144v2 Announce Type: replace Abstract: Membership inference (MIA) on language models is usually summarised by aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines can separate members from non-members using surface text alone.
By Victor Maricato
arXiv:2606. 25449v1 Announce Type: cross Abstract: A language model's memory can be worse than having no memory at all.
By Alex Kwon
arXiv:2610.17719v1 Announce Type: new
Abstract: Agents build knowledge graphs as memory, and building them has become easy. Maintaining them has not. When a source is retracted, overruled, or erased,...
By Vishisht Choudhary
The paper investigates how short memory representations of large‑language‑model (LLM) group discussions affect final decision quality. It introduces CrossProfile, a benchmark of 480 hidden‑profile tasks that separates supporting, decoy, and irrelevant facts while keeping their wording constant. Experiments with Qwen2.5‑7B, Llama‑3.1‑8B, and DeepSeek models show that heuristics based on rarity or relevance both fail in predictable ways, and that the decision model’s social‑proof bias further degrades accuracy. Removing attribution tags or randomizing them improves performance, yet natural transcripts still lose more evidence than no notes at all.
By Anouar Imel, Imad Charradi, Abderrahman Sghir, Khalil El Khammar