The paper introduces a benchmark for evaluating whether off‑the‑shelf small language models (SLMs) can reliably perform microtasks that support a large language model (LLM) planner, such as auto‑approving shell commands, writing memory, selecting tools, and ranking past turns. Using fixed prompts and confidence‑interval‑aware eligibility thresholds, the authors test several Qwen3 models (0.6/1.7/4/8 B) in FP16 with no tuning and find that none of the 16 configurations meet the eligibility criteria. Quantization to 4‑bit precision further degrades performance, with the eligibility gap tracking model size rather than precision, and the issue persists across different models (e.g., Llama‑3.x) and prompt variations.
By Jundong Hu, Shekar Ramachandran
arXiv:2610.00279v1 Announce Type: new
Abstract: The segmentation of anatomical structures in medical images and particularly in MRI scans, is essential for clinical diagnosis and monitoring disease p...
By Eirini Cholopoulou, Dimitrios E. Diamantis, Dimitris K. Iakovidis
Agent evaluations increasingly benchmark LLMs, but rankings can be swayed by evaluation conditions such as scaffolds or tasks, making reliability claim‑dependent. A Bayesian variance‑decomposition framework applied to 22 benchmarks shows that reliability varies with the measurement goal: fixed model‑scaffold systems rank reliably, while underlying‑model rankings are less stable. Scaffold choice can alter conclusions, and adding more tasks only modestly improves reliability when scaffold coverage is limited; however, pooling diverse benchmarks can substantially raise cross‑task ranking reliability and reduce cost.
By Michael Hardy, Ruhana Azam, Anka Reuel, Mykel Kochenderfer, Sanmi Koyejo
The paper introduces a method to purify LoRA-tuned large language models (LLMs) against backdoor attacks without relying on trigger knowledge, clean references, or retraining. By extracting high‑fidelity backdoor directions and projecting LoRA updates onto orthogonal null spaces in input and output channels, the approach reduces attack success rates from nearly 100% to under 10%. Experiments demonstrate that this null‑space projection preserves both the base model’s general capabilities and the new downstream skills learned through the adapter across various tasks.
By Jianwei Li, Jung-Eun Kim
arXiv:2610.00693v1 Announce Type: new
Abstract: Federated learning (FL) has recently attracted increasing attention in remote sensing (RS) since it enables collaborative model training across decentr...
By Bar{\i}\c{s} B\"uy\"ukta\c{s}, Beg\"um Demir
R-GroundBench is a new diagnostic benchmark for evaluating AI models on R‑group grounding in Markush molecular editing, derived from real pharmaceutical patents. It includes a Multiple‑Choice VQA track with varying difficulty and modality splits, as well as an open‑ended Generation track. Experiments show a large performance gap: models score over 90% on easy VQA but drop to 56–66% on hard VQA, and generation exact match stays below 20% (and under 8% with visual input).
By Xin Wang, Zichuan Ying, Xinna Lin, Junqi Zhang, Hanyi Xiong, Tianyu Gao, Hairong Zhang, Qixiang Hua, Botian Shi, Zhenhailong Wang, Kaicheng Yu
arXiv:2610.00812v1 Announce Type: cross
Abstract: Video generation has rapidly progressed from short, low-quality clips to high-resolution, long-duration sequences with complex spatiotemporal dynamic...
By Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, Tianzhi Li, Fei Tao, Pooyan Fazli
arXiv:2610.00851v1 Announce Type: new
Abstract: Open Set Recognition (OSR) aims to enable models to accurately classify known classes while rejecting samples from unseen classes. A key challenge in O...
By Thiru Thillai Nadarasar Bahavan, Yu Xia, Sachith Seneviratne, Saman Halgamuge
ReHoPER is an inference‑only, zero‑shot method that enhances large language models’ reasoning by generating and answering intermediate questions along multiple paths before producing a final answer. It plans a horizon of candidate intermediate questions, selects one to answer, and replans based on the updated history. The approach is task‑agnostic, using generic instructions across datasets and models without labeled data or task‑specific prompt design, and it outperforms strong baselines on several datasets, notably achieving the largest gains on the new iLLC benchmark for compositional reasoning.
By Saeed Ahmadnia, Cornelia Caragea
arXiv:2610.01428v1 Announce Type: cross
Abstract: Generalization in large language models (LLMs) is the ability to produce consistent and semantically stable outputs when the same input is expressed...
By Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi
The paper investigates how reinforcement learning post‑training of large language models (LLMs) tends to sharpen existing behaviors, improving single‑shot accuracy but reducing solution coverage. It shows that pre‑trained LLMs, when paired with a lightweight inference harness, can outperform post‑trained models in coverage for agentic tasks that require multi‑turn tool use. The authors introduce the Sharpening Tax metric to quantify this trade‑off, analyze its prevalence across 14 model pairs and 42 benchmark cases, and propose a Bayesian sampler, PTGS, that mitigates the tax by adapting sampling temperature to prompt difficulty.
By Changdae Oh, Qi Zeng, Qi Qi, Andrey Zhmoginov, Deren Lei, Yun He, Hoang Phan, Hangoo Kang, Azalia Mirhoseini, Sharon Li
The paper introduces Graph-Informed Semantic IDs (GrIS), a framework that reframes Semantic ID construction as a recursive clustering problem on a graph combining semantic content and collaborative signals. GrIS generalises previous methods like RQ-VAE and RQ-KMeans by allowing explicit graph construction and hierarchical partitioning, and presents two implementations: RecDMoN and RQ-GAE. Experiments on real-world datasets show that GrIS outperforms collaborative-filtering aware state‑of‑the‑art models, achieving up to a 52% increase in Hit@10.
By Aleksei Medvedev, Alejandro Ariza-Casabona, Steven Derby, Gonzalo Fiz Pontiveros, Xinyang Shao, Florian Spiess
arXiv:2610.01546v1 Announce Type: cross
Abstract: Primal-dual hybrid gradient (PDHG) methods solve large-scale linear programs (LPs) using GPU-friendly matrix-vector products and projections, but the...
By Jinhwan Sul, Alex Oshin, Evangelos A. Theodorou
arXiv:2610.01670v1 Announce Type: new
Abstract: Multimodal large language models (MLLMs) are increasingly used as automated judges for instruction-based image editing and as reward signals for model...
By Yuan Huang, Zirui Song, Xiuying Chen
arXiv:2610.02000v1 Announce Type: new
Abstract: Adverse conditions such as rain, snow, fog, and dust remain challenging for camera-based perception in autonomous driving. We study multi-class weather...
By Hossein Maghsoumi, George Atia, Yaser P. Fallah
arXiv:2610.02117v1 Announce Type: cross
Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...
By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris
arXiv:2610.02150v1 Announce Type: cross
Abstract: Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods...
By Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, Haibo Jin, Yijia Xiao, Wenke Lee, B. Aditya Prakash, Haohan Wang
arXiv:2502.04892v2 Announce Type: replace-cross
Abstract: Learning robust representations from functional magnetic resonance imaging (fMRI) is fundamentally challenged by the temporal irregularity an...
By Joonhyeong Park, Byoungwoo Park, Chang-Bae Bang, Jungwon Choi, Hyungjin Chung, Byung-Hoon Kim, Juho Lee
The article discusses how predictive benchmarking—evaluating machine learning models by their performance and ranking—serves as a core method in machine learning research. It argues that benchmark scores only reflect performance on specific datasets and learning problems, and that drawing broader scientific conclusions requires explicit assumptions. By adapting concepts from psychological validity theory, the authors propose validity conditions to make these assumptions clear, and demonstrate their application in two case studies (ImageNet and the Fragile Families Challenge) to illustrate how benchmark results can inform inferences about research progress and limits of predictability.
By Timo Freiesleben, Sebastian Zezulka
arXiv:2510.24046v2 Announce Type: replace-cross
Abstract: Existing tabular data generation methods primarily focus on matching statistical distributions between real and synthetic data, often overloo...
By Tu Anh Hoang Nguyen, Dang Nguyen, Tri-Nhan Vo, Thuc Duy Le, Trung Le, Sunil Gupta