The paper introduces LoDA, a LoRA-based continual learning method that decomposes update spaces into task‑shared and truly task‑specific subspaces using energy‑based objectives. By fixing down‑projections and learning up‑projections with Gradient‑Aligned Optimization, LoDA recalibrates general updates in closed form after each task, aiming to improve knowledge sharing and isolation. Experiments show that LoDA outperforms existing CL methods.
By Lingfeng He, De Cheng, Huaijie Wang, Xi Yang, Nannan Wang, Xinbo Gao
SWEPatchTest is a benchmark that evaluates software tests by measuring their ability to reproduce issues, accept correct patches, and reject incorrect ones across 619 tasks and 82,579 candidate patches. It introduces the Joint Validation Rate metric, revealing that even the best configurations only achieve 9.21% success when all three criteria are met. Using these validation scores to select demonstrations for supervised fine‑tuning can roughly triple the Joint Validation Rate and enhance downstream repair performance on multiple SWE‑bench datasets.
By Hua Lu, Pengcheng Wang, Han Li, Zichen Tian, Yanyu Chen, Huang He, Bingjin Chen
The paper introduces Predictability-Aware Canvas Selection and Supervision (PACS), a supervised fine‑tuning method for diffusion language models that aligns training with inference by using the model’s own predictability to determine which tokens are visible and when they are supervised. PACS selects the attention‑visible input length for each example and supervises reference tokens just before they become confident, addressing mismatches caused by random masking. Experiments on reasoning and code‑generation benchmarks show PACS consistently outperforms standard SFT, achieving up to 12.84 percentage‑point gains on individual tasks.
By Miao Li, Hanyang Jiang, Hengyu Fu, Baihe Huang, Yating Liu, Yuhang Cai, Changkun Guan, Zhongzhu Chen, Jiecheng Lu, Michael A. Boateng, Pascal Van Hentenryck
DriveWLA is a unified World‑Language‑Action model that integrates semantic understanding, future scene dynamics, and trajectory generation for autonomous driving. It employs a Mixture‑of‑Transformers architecture with three experts—World, Language, and Action—connected by Cross‑Expert Attention to align semantic and dynamic representations. After reinforcement learning refinement, DriveWLA outperforms existing VLM‑based and world‑model‑based planners on NAVSIM benchmarks and generalizes well to other datasets without fine‑tuning.
By Shuzheng Wang, Haoxuan Li, Shengzhen Li, Chenxi Wu, Yajing Sun, Lu Hou, Wenyu Liu, Xinggang Wang
SpecCloak investigates how presenting coding tasks as engineering specifications—via the PseudoSpec template—affects large language models’ refusal to generate harmful code. Experiments on five open-weight models show that specification prompting significantly lowers refusal rates even when risk keywords remain, shifting internal representations toward code-output contexts. Building on this, SpecCloak combines PseudoSpec with discriminator-guided optimization to achieve a 28.91% attack success rate, while SpecReveal offers a defense that detects such attacks with 96.88% accuracy without fine‑tuning the target model.
By Wenbin Chen, Jiexin Wang, Guanting Wu, Jiaxi Liang, Yi Cai
MERIT (Multimodal Evidence Reasoning over Image Tables) is a three‑stage framework designed to enable language models to reason over table images, which preserve visual cues and structural details that serialized textual tables often lose. The approach first adapts a table‑specific encoder through reconstruction, localization, and color‑conditioned supervision to address visual evidence acquisition. It then fine‑tunes the model with supervised evidence selection and reasoning, followed by reinforcement learning that rewards evidence correctness, calculation, final answers, and output format, thereby improving retrieval and computational accuracy in table question answering and fact verification tasks.
By Haoyang Jiang, Zhuotong Fan, Mingyang Yi, Ju Fan, Lanqing Hu, Xiao Zan
The paper introduces inLoRA-MINT, an extension of the LoRA-MINT framework that uses intermediate model representations—such as activations and parameter-dependent signals—to perform Membership Inference Tests on large language models fine‑tuned with LoRA. By leveraging these representations, the method can distinguish training from non‑training samples without full parameter access, achieving 95–98% accuracy across four LLMs (Qwen, Phi‑4, DeepSeek‑V3, LLaMA 3.2‑3B) on medical, mathematical, and academic reasoning datasets. The approach offers a scalable way to audit sensitive training data and supports compliance with emerging privacy and AI governance regulations.
By Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado
The paper investigates how to best utilize demonstration quality during vision‑language‑action (VLA) pretraining to boost downstream task performance. It compares weighting, filtering, and conditioning strategies across different reward definitions and label granularities, using 5,000 suboptimal human demonstrations and 24 target tasks while keeping demonstration sets and training budgets matched. The study introduces AXIS‑Bench, a benchmark built on crowdsourced simulation demonstrations that allows recomputation of quality labels and replayable trajectories for both simulation and real‑world tasks, showing that simulation pretraining can raise real‑world success rates from 9.6% to 31.7%.
By Mingxuan Yan, Litian Gong, Dihong Huang, Zhixu Li, Tianyu Zhang, Wenqian Zhang, Hefeifei Jiang, Peihao Li, Jian Zhang, Mengfei Zhao, Yikai Tang, Hai Zhai, Zehao Wang, Ruijian Liang, Yanjia Huang, Lin Shao, Changliu Liu, Kaiyu Hang, Zhiwen Fan, Masayoshi Tomizuka, Jianfei Yang, Jiachen Li
ScoringBench is an open, extensible benchmark that evaluates tabular regression models using a comprehensive set of proper scoring rules—such as weighted CRPS, energy scores, and interval scores—alongside traditional point metrics like RMSE. It covers 101 datasets from diverse domains and offers two ranking protocols: an ordinal Demsar/autorank approach and a magnitude‑preserving z‑score ranking. Experiments show that model rankings vary with the chosen scoring rule, revealing that evaluation metric selection is a core modeling decision, especially when different error types have varying consequences.
By Jonas Landsgesell, Pascal Knoll, Tizian Wenzel
arXiv:2610.17727v2 Announce Type: cross
Abstract: Agent skills turn natural-language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look d...
By Jerrold Huang, Junwei Quan, Saiyue Lyu, Zihan Guo, Yingqiang Ge
arXiv:2610.16761v1 Announce Type: new
Abstract: LLM-based information systems need safety guardrails. Most guardrails classify the prompt alone. If the prompt looks harmful, the system rejects it bef...
By Ram Vikas Mishra, Shashank Aggarwal, Jahnavi Kumar, Amit Awekar
arXiv:2610.17062v1 Announce Type: cross
Abstract: We propose a mitigation to reward hacking in reinforcement learning. Our setup involves a misspecified reward function that gives rise to unintended...
By Jake Ward, Shawn Hu, Aria Wong, Nathan Hu, Bryce Woodworth, Alexander Matt Turner, Alex Cloud
arXiv:2607.01455v4 Announce Type: replace-cross
Abstract: Language models learn continuous programs over discrete symbols, with the embedding table and LM-head acting as the read/write interface betw...
By Kathan Shah
arXiv:2610.15056v1 Announce Type: cross
Abstract: Large language models can post-edit ASR by using linguistic context to compare N-best hypotheses and rewrite the transcript when needed, often improv...
By Shiran Liu, Gaohui Xu, Sergio Burdisso, Thibault Ba\~neras-Roux, Shashi Kumar, Petr Motlicek
arXiv:2610.15092v1 Announce Type: cross
Abstract: Quantization plays a crucial role in efficiently deploying deep learning models on resource-constrained devices. Post-training quantization does not...
By Reem Abdel-Salam, Ahmed H. Abdel-Gawad, Amr G. Wassal
arXiv:2610.19044v1 Announce Type: cross
Abstract: Tiny robots have limited computation and memory, yet visual navigation systems often maintain more state than a task requires. We ask whether persist...
By Naitri Rajyaguru, Aryan Tajne, Stephen Jardim, Cornelia Fermuller, Yiannis Aloimonos
arXiv:2610.16782v1 Announce Type: new
Abstract: Industrial manipulation in flexible manufacturing, e.g., high-mix, low-volume (HMLV) electronics assembly, requires robot policies to adapt to changing...
By Ming Shin Cindy Sung, Duan Leijiahao, Haoxiang Wang, Boning Tan, Liangna Fu, Chao Tan, Yafu Li, Yulong Chen, Chaoqun Wan, Zhen Huang, Shuxian Liang, Xian-Sheng Hua
arXiv:2610.16946v1 Announce Type: cross
Abstract: Descriptions of how people use AI are becoming training data for the next generation of models. We ask whether what a model reads about its users cha...
By Harsh Kumar, George Eilender, Ashton Anderson
arXiv:2610.15492v1 Announce Type: new
Abstract: Skill internalization aims to convert externally specified procedural skills into persistent model capabilities, allowing language-model agents to exec...
By Haomiao Qiu, Yining Tang, Hongyi Wan, Changfeng Sun, Ziyue Qiao, Miao Zhang
arXiv:2610.14709v1 Announce Type: cross
Abstract: Large language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommen...
By Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Weiqiang Hang, Luqing Wang, Zejian Wang, Zirui Tu, Jingsheng Mao, Shaobo Hu, Yang Luo, Jiale Guo, Jin Zhang, Yudan Liu, Zhaokai Luo, Ren Huan, Yao Hu, Chengyue Jiang, Wenjie Li, Chuan Mu