CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.
By Eunju Park, Uiwon Hwang
FastJEV addresses redundancy in JEV models by reusing context and intermediate states during candidate evaluation, eliminating repeated token matching and full backbone execution. It introduces shared context anchoring and candidate prefix sharing to preserve necessary states for branching, and applies decision‑guided pruning based on score changes from a small unlabeled set. Experiments on OmniJev models across five benchmarks show that FastJEV can reduce candidate depth by up to ~45% while maintaining over 94% of original task scores.
By Jie Ma, Jie Gao, Yihang Liu, Zhike Qiu, Junle Li, Chongyi Zhuang, Jiayi Ji, Xiaoshuai Sun
arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.
By Gizem Y\"uce, Giorgos Nikolaou, Nicolas Flammarion
arXiv:2604.16656v2 Announce Type: replace
Abstract: All languages are equal; when it comes to tokenization, some are more equal than others. Tokens are the hidden currency that dictates the cost and...
By Maitrey Mehta, Nishant Subramani, Zhichao Xu, Ashim Gupta, Vivek Srikumar
arXiv:2602.22453v4 Announce Type: replace
Abstract: Retrieval heads, a subset of attention heads in Transformers, were studied in English, showing its crucial role in retrieving information from the...
By Shaswat Patel, Vishvesh Trivedi, Yue Han, Yihuai Hong, Eunsol Choi
Raincheck is a framework that defers multimodal prefill for large language models, allowing initial token selection to be decoupled from later information needs. It preserves unselected tokens and progressively prefills them during decoding, using unused compute capacity and scheduling gaps. Experiments on Qwen3-VL-8B and Qwen3.5-9B show that Raincheck recovers a significant portion of quality lost to hard pruning and reduces first-turn time-to-first-token by up to 24.1% with minimal impact on token generation speed.
By Wenjie Huang, Jun Zhang, Hongyaoxing Gu, Haoyang Huang, Tianqi Xu, Xinyue Hao, Xinyi Hu, Xinle Du, Jun Dai, Shuang Ge, Chen Lei, Yue Li, Mingcheng Wan
MAxBench is a geometry‑agnostic benchmark for evaluating how well language models recover multinomial concept representations. The study compares ten localization methods across five geometry types, six concepts, and four models, finding that affine subspaces generally steer more reliably and recall more instances than rank‑one or linear subspaces. The results also show that manifold steering can match the best methods when applicable, and that no method consistently outperforms prompting for these complex concepts.
By Divya Appapogu, Freya Behrens, Yonatan Belinkov, Aaron Mueller
arXiv:2605.16893v2 Announce Type: replace
Abstract: Recent studies have introduced memory-augmented architectures that decouple knowledge storage from neural computation, enabling more direct access...
By Yuwen Qu, Wenhui Dong, Camil Hamami, Ming Zhao, Chenyang Si, Caifeng Shan
arXiv:2606. 15521v1 Announce Type: cross Abstract: Tokenization introduces representational redundancy: under a fixed token vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string.
By Kanishk Jain, Matthew Day, Tankut Can
arXiv:2610.18604v1 Announce Type: cross
Abstract: 3D part assembly aims to reconstruct a complete shape from a set of unassembled parts by predicting a rigid SE(3) transformation for each part. Becau...
By Zhouhan Zhong, Qiuhong Shen, Xingyi Yang, Jinyi Xu, Xinchao Wang
The paper investigates which linguistic operations can be replaced by latent task representations instead of in‑context prompts. A 2.6 M‑parameter network learns to encode the geometry of a few‑shot support set and applies an additive update to a frozen GPT‑2 model, achieving zero‑shot performance comparable to or better than 10‑shot in‑context learning for forward inflection and lemmatization. However, for arbitrary pairings such as antonymy, the latent approach only reaches about half the performance of in‑context learning, indicating that productive rules amortize well while memorized pairings do not.
By Gunmay Jhingran
arXiv:2608. 05162v1 Announce Type: cross Abstract: Pooling is a consequential but under-examined design choice in decoder-only concept representation work: practitioners must collapse token-level hidden states into a passage-level vector, yet no shared protocol exists for comparing this choice across concepts, models, and tasks.
By Ayushi Agarwal