The paper introduces CARPAS, a new task that dynamically refines user-provided aspects for aspect-based summarization in large language models (LLMs). It presents three new datasets and evaluates four prompting strategies, finding that LLMs tend to over-generate aspects, leading to overly long and misaligned summaries. To address this, the authors propose a two-stage framework that first generates lightweight scope guidance before aspect refinement and summarization, which improves focus, reduces over-generation, and enhances performance across all datasets.
By Yong-En Tian, Yu-Chien Tang, An-Zi Yen, Wen-Chih Peng
AutoIndexer is a generative model that trains causal models on chains of edits, enabling flexible post‑generation modifications such as token insertions, substitutions, and deletions. It introduces marker tokens for opening and closing edits and moves a cursor through the token sequence, mimicking a text editor. By perturbing ground‑truth labels with inverse edit operations during training, AutoIndexer becomes robust to erroneous context and can perform editing tasks like code correction while maintaining performance on standard LLM benchmarks.
By Shu Ishida, Aliasghar Khani, Tianyu Zhang, The Cong Luong, James Seale Smith, Adam Gaier
DynEdit is a dynamic framework for editing long-sequence knowledge in large language models. It uses token-level entropy to guide non-uniform sliding windows, focusing on high-entropy, information-dense regions that are harder to edit. Experiments on three benchmarks and two backbone models show DynEdit outperforms baselines, achieving up to +14.0 BLEU points on challenging paraphrase queries and near-saturated semantic scores across diverse knowledge domains.
By Jinhu Fu, Yan Bai, Yihang Lou, Li Sun, Jiahong Wu, Xiangxiang Chu, Sen Su
arXiv:2609.20830v1 Announce Type: new
Abstract: Revision-capable generation is appealing because it can insert or revise earlier content, but many non-autoregressive and edit-based approaches obtain...
By Sean Diab
Causal Edit Serialization (CES) is a framework that enables autoregressive language models to perform revisions—such as insertion, deletion, replacement, copying, and movement—within the same causal token stream. CES uses special pointer tokens paired with predicted source addresses, a lightweight localization head, and dual-index rotary position encoding to localize edits while preserving standard causal attention. Experiments show that CES models can execute complete edits with only a modest increase in clean-text bits per byte, and that post‑training with supervised or reinforcement learning can produce beneficial grammatical error corrections.
By Louis Robinson
RangeSWA is a training‑free method designed to improve length extrapolation in continuous diffusion language models. It combines local sliding‑window attention, remote block summaries, and dynamic content retrieval to maintain local detail while accessing long‑range context. The approach also includes positional remapping, conditioning attention correction, and denoising‑stage‑adaptive window scheduling to mitigate generation degradation, resulting in lower perplexity and fewer repetitions in long‑text generation.
By Guangfu Guo, Xiaoqian Lu, Linsey Pang, Weiran Yao, Haolin Chen, Long Cheng, Kunpeng Liu