The paper studies where to place task‑specific adapters in a vision transformer to balance storage growth and accuracy. Training all contiguous four‑block placements shows an inverted‑U accuracy curve, peaking at intermediate depths, while simple weight or activation metrics favor the deepest blocks. A neuroscience‑inspired method, LS‑B, uses frozen fMRI readouts of human visual areas to select blocks whose responses vary most across tasks, yielding backbone‑specific allocations that match or exceed the best placements found by search and use only 60% of the adapter storage while staying within 1.5 percentage points of full accuracy.
By Yuan Huang, Zihan Chen, Runbin Zhang, Hongwei Ding, Changzeng Fu, Shiqi Zhao
arXiv:2610.01712v1 Announce Type: cross
Abstract: In-context learning (ICL) enables a pretrained model to infer a task from demonstrations without updating its parameters. While much of the existing...
By Haotian Gu, Yizhou Xu, Lenka Zdeborov\'a
arXiv:2610.02015v1 Announce Type: cross
Abstract: Recent advances in LLM reasoning models---driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)-...
By Michael Sullivan, Alexander Koller
arXiv:2610.02039v1 Announce Type: cross
Abstract: Recent years have witnessed the rapid adoption of reinforcement learning (RL) in large language model (LLM) post-training, with substantial gains in...
By Yafei Zhang, Songshuo Lu, Sicong Liao, Zhi Chen, Yaohua Tang
arXiv:2610.02126v1 Announce Type: cross
Abstract: We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of...
By Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes
arXiv:2610.00006v1 Announce Type: new
Abstract: Pretrained Vision Transformers encode whether two image patches belong to the same object. This IsSameObject signal is decodable from frozen patch embe...
By Mayank Singal
arXiv:2610.00024v1 Announce Type: new
Abstract: Across three vision-language model architectures (LLaVA-1.5-7B, Qwen2.5-VL-7B, InternVL3-8B), we report a universal negative finding for mid-layer inte...
By Genpei Zhang
The paper introduces a benchmark for evaluating whether off‑the‑shelf small language models (SLMs) can reliably perform microtasks that support a large language model (LLM) planner, such as auto‑approving shell commands, writing memory, selecting tools, and ranking past turns. Using fixed prompts and confidence‑interval‑aware eligibility thresholds, the authors test several Qwen3 models (0.6/1.7/4/8 B) in FP16 with no tuning and find that none of the 16 configurations meet the eligibility criteria. Quantization to 4‑bit precision further degrades performance, with the eligibility gap tracking model size rather than precision, and the issue persists across different models (e.g., Llama‑3.x) and prompt variations.
By Jundong Hu, Shekar Ramachandran
arXiv:2610.00562v1 Announce Type: cross
Abstract: Longitudinal clinical reasoning requires large language models (LLMs) to identify and integrate relevant evidence distributed across extended patient...
By Taye Akinrele, Noorbakhsh Amiri Golilarz, Subash Neupane, Sudip Mittal, Shahram Rahimi
arXiv:2610.00571v1 Announce Type: cross
Abstract: Large reasoning models (LRMs) have achieved substantial improvements in solving complex mathematical problems, but often produce lengthy, repetitive,...
By Barproda Halder, Qiuyi Zhang, Sanghamitra Dutta
Agent evaluations increasingly benchmark LLMs, but rankings can be swayed by evaluation conditions such as scaffolds or tasks, making reliability claim‑dependent. A Bayesian variance‑decomposition framework applied to 22 benchmarks shows that reliability varies with the measurement goal: fixed model‑scaffold systems rank reliably, while underlying‑model rankings are less stable. Scaffold choice can alter conclusions, and adding more tasks only modestly improves reliability when scaffold coverage is limited; however, pooling diverse benchmarks can substantially raise cross‑task ranking reliability and reduce cost.
By Michael Hardy, Ruhana Azam, Anka Reuel, Mykel Kochenderfer, Sanmi Koyejo
The paper introduces a method to purify LoRA-tuned large language models (LLMs) against backdoor attacks without relying on trigger knowledge, clean references, or retraining. By extracting high‑fidelity backdoor directions and projecting LoRA updates onto orthogonal null spaces in input and output channels, the approach reduces attack success rates from nearly 100% to under 10%. Experiments demonstrate that this null‑space projection preserves both the base model’s general capabilities and the new downstream skills learned through the adapter across various tasks.
By Jianwei Li, Jung-Eun Kim
arXiv:2610.00694v1 Announce Type: cross
Abstract: Compression reports summarize how far a compressed language model moved from the dense one, usually by a KL divergence; a deployment that relies on t...
By Beatriz Almeida Felicio
arXiv:2610.00922v1 Announce Type: new
Abstract: Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict...
By Jungmin Lee, Niamat Ullah, Yoseob Han
ReHoPER is an inference‑only, zero‑shot method that enhances large language models’ reasoning by generating and answering intermediate questions along multiple paths before producing a final answer. It plans a horizon of candidate intermediate questions, selects one to answer, and replans based on the updated history. The approach is task‑agnostic, using generic instructions across datasets and models without labeled data or task‑specific prompt design, and it outperforms strong baselines on several datasets, notably achieving the largest gains on the new iLLC benchmark for compositional reasoning.
By Saeed Ahmadnia, Cornelia Caragea
arXiv:2610.00997v1 Announce Type: cross
Abstract: Knowledge distillation aims to transfer the factual knowledge of large language models to smaller models for efficient deployment. Yet a teacher may...
By Jungseob Lee, Sugyeong Eo, Seongtae Hong, Seungyoon Lee, Chanjun Park, Jaehyung Seo, Heuiseok Lim
arXiv:2610.01054v1 Announce Type: cross
Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requi...
By Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang
arXiv:2610.01428v1 Announce Type: cross
Abstract: Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed...
By Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi
The paper investigates how reinforcement learning post‑training of large language models (LLMs) tends to sharpen existing behaviors, improving single‑shot accuracy but reducing solution coverage. It shows that pre‑trained LLMs, when paired with a lightweight inference harness, can outperform post‑trained models in coverage for agentic tasks that require multi‑turn tool use. The authors introduce the Sharpening Tax metric to quantify this trade‑off, analyze its prevalence across 14 model pairs and 42 benchmark cases, and propose a Bayesian sampler, PTGS, that mitigates the tax by adapting sampling temperature to prompt difficulty.
By Changdae Oh, Qi Zeng, Qi Qi, Andrey Zhmoginov, Deren Lei, Yun He, Hoang Phan, Hangoo Kang, Azalia Mirhoseini, Sharon Li
arXiv:2610.01569v1 Announce Type: cross
Abstract: Unmanned aerial vehicle (UAV) swarms increasingly rely on language-model agents to provide adaptive mission-level reasoning in uncertain environments...
By Andrea Iannoli, Ivan Zyrianoff, Angelo Trotta, Lorenzo Gigli, Marco Di Felice