Attention-based rerankers use a null-query calibration step to reduce positional and structural bias, but this step can inadvertently remove useful signal when prompts contain detailed instructions, constraints, personas, or demonstrations. The study demonstrates that such calibration harms performance on instruction-heavy tasks and introduces an interpolated null calibration that selectively limits instruction influence on the null baseline. This modification restores or surpasses the effectiveness of standard rerankers on instruction-heavy tasks while maintaining benefits when the null pass remains relevance-agnostic, and shows that in-context demonstrations can improve reranking with minimal calibration interference.
arXiv:2609.13141v1 Announce Type: new Abstract: Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by selecting a small set of context u...
arXiv:2606.02737v2 Announce Type: replace-cross Abstract: Dense retrieval compresses a passage into a single vector, but this compression is positionally skewed: early content dominates the embedding...
arXiv:2606. 04434v1 Announce Type: cross Abstract: Multimodal In-Context Learning (ICL) has emerged as a practical inference paradigm for Multimodal Large Language Models, where a small set of interleaved image-text In-Context Demonstrations (ICDs) conditions the model to solve new tasks.
The paper investigates how to fine‑tune language models for sparse and continual reinforcement learning by ranking and partitioning attention heads. For sparse learning, heads are ranked by gradient norm to identify a small, effective subset for training. For continual learning, heads are partitioned across tasks to reduce interference and preserve prior knowledge. Empirical results on math, instruction‑following, and coding tasks show that tuning higher‑ranked heads outperforms lower‑ranked ones and matches denser baselines, while head partitioning improves retention without sacrificing adaptation to new tasks.
arXiv:2511. 16107v3 Announce Type: replace-cross Abstract: Visual in-context learning (VICL) solves visual tasks by conditioning on a few input-output demonstrations without any model training.
arXiv:2606. 24477v1 Announce Type: cross Abstract: Video large language models (LLMs) are often constrained by computation and memory budgets, leading them to use reduced frame rates and spatial resolutions, which may cause them to miss critical information for question answering (QA).
arXiv:2502. 08363v3 Announce Type: replace-cross Abstract: We present Top-Theta (Top-$\theta$) Attention, a training-free method for sparsifying transformer attention during inference.
arXiv:2608.23921v1 Announce Type: new Abstract: Recent Vision-Language Models encode high-resolution images into long visual token sequences, incurring prohibitive prefill costs. To compress them, ex...
The paper investigates where to insert Cross‑Attention Layer (CAL) blocks between a system prompt and text in a frozen causal‑decoder backbone. A sweep on a 1.5B model shows that later placements of CALs are generally more effective and parameter‑efficient, while an 8B scaling study confirms that the impact remains task‑dependent, affecting instruction‑following and security behavior without significantly harming general performance. These results highlight placement as a key design choice when injecting system‑prompt information via cross‑attention.
arXiv:2609.37581v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visu...
arXiv:2609.14127v1 Announce Type: cross Abstract: We derive exact local responses for attention interventions, allowing candidate edits to be scored from a cached baseline and one backward pass. The...
arXiv:2605. 04495v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on evidence ranking to determine what information is exposed to the generator, yet existing retrieval and reranking methods primarily estimate query--document relevance.