The paper introduces the concept of a prior barrier to measure how well pretrained large language models support different concepts during supervised fine‑tuning (SFT). It shows that these barriers follow a long‑tail distribution, causing head concepts to be easier to learn than tail concepts. To address this, the authors propose PASS, an adaptive instruction‑selection method that allocates limited supervision budget toward under‑covered concepts, outperforming seven state‑of‑the‑art methods across multiple settings.
By Haohui Wang, Jiahao Xu, Wangzhi Zhan, Tong Zeng, Dongqi Fu, Hong Li, Swastik Roy, Naren Ramakrishnan, Chris North, Jian Kang, Yujun Yan, Dawei Zhou
C-LoRA introduces a continual learning approach for pre‑trained visual models that uses a single shared Low‑Rank Adaptation (LoRA) adapter. It employs a learnable routing matrix split into a stability component (R_base) and a plasticity component (R_delta) to balance preserving past knowledge and adapting to new tasks. This design eliminates the need for expanding modules or complex inference fusion while maintaining competitive performance on multiple benchmarks.
By Xin Zhang, Liang Bai, Xian Yang
Opera is a verbal critic framework designed for long‑horizon coding agents, treating each correction as a persistent note that is followed until the problem is resolved. It uses periodic and event‑driven triggers to diagnose issues, audits feedback against visible evidence, and tracks subsequent actions to distinguish compliance from true resolution. In tests, Opera improves the resolve rate of non‑critic agents by up to 12.4, 15.0, and 8.9 percentage points on three benchmarks and provides on‑policy training data that boosts a model’s performance without requiring a critic at inference time.
By Kai Mei, Zhiyuan Hu, Yutong Dai, Juntao Tan, Yifan Zhang, Dingjie Song, Dimitris N. Metaxas, Silvio Savarese, Ran Xu, Zeyuan Chen
The paper describes a system for the PlurVA-LLM 2026 Shared Task Track‑1, which aims at pluralistic value alignment in China, Indonesia, and Sri Lanka. The authors fine‑tuned Llama 3.1 8B Instruct with 4‑bit QLoRA and applied data‑specific techniques: option‑permutation augmentation for Chinese data, annotator vote expansion for Indonesian data, and binary reformulation with SinhalaMMLU augmentation plus conditional threshold calibration for Sri Lankan data. The resulting model achieved macro‑average accuracy of 0.805, with per‑language accuracies of 0.785 (Chinese), 0.715 (Indonesian), and 0.916 (Sri Lankan).
By Vihindi Kotalawala, Nevidu Jayatilleke
The paper introduces Latent-Attention Masked Autoencoder (LAMAE), a multimodal masked autoencoder that learns patient-level representations by exchanging information directly in the latent space through a shared latent-attention module. LAMAE integrates complementary modalities—electrocardiogram, echocardiography, and chest X-rays—within a study-view-entity hierarchy, enabling aggregation of variable observations and handling missing modalities. Pretraining on over 500,000 MIMIC-IV hospital stays, LAMAE outperforms modality-specific pretraining and strong contrastive and vision-language baselines on multimodal hospital-stay tasks such as in-hospital mortality, ICD-10 and DRG coding, and length of stay, while remaining competitive on unimodal tasks.
By Andrea Agostini, Simon B\"ohi, Moritz Vandenhirtz, Samuel Ruiperez-Campillo, Max Kr\"ahenmann, Silke M\"uhlstedt, Irene Cannistraci, Ece \"Ozkan Elsen, Julia E. Vogt, Thomas M. Sutter
Agentic Critical Training (ACT) is a reinforcement learning approach that trains language‑model agents to judge actions directly by pairing expert actions with alternatives and rewarding correct selection. Unlike imitation learning or self‑reflection, ACT does not require reference rationales and can be reused across model sizes. Experiments on Qwen3‑8B and Olmo‑3‑7B‑Instruct show that ACT improves performance on ALFWorld, WebShop, ScienceWorld, and outperforms supervised reflection baselines, with significant gains over IL and IL→RL baselines.
By Weize Liu, Minghui Liu, Sy-Tuyen Ho, Yongkyun Lee, Andrew Adams Schoen, Souradip Chakraborty, Xiyao Wang, Furong Huang
The paper investigates whether Multimodal Large Language Models (MLLMs) can reason about visual persuasion. It finds that current models often shortcut by citing obvious visual elements instead of truly reasoning about diverse cues, and proposes a fine‑tuning method using multi‑perspective rationales along with an evaluation framework that separates prediction performance from rationale faithfulness. The study shows that while fine‑tuning improves prediction, higher accuracy does not guarantee more faithful reasoning, underscoring the need for distinct evaluation of effectiveness and faithfulness.
By Naeun Lee, Hyunjong Kim, Sunghwan Choi, Injin Kong, Yohan Jo
WorldCraft is an autoregressive video world model that enables both camera navigation and persistent object-trajectory control in interactive video environments. It introduces Normalized World Trajectory (NWT) to anchor user-specified 2D motion paths in a world coordinate system, and Trajectory-Anchored State Persistence (TASP) to maintain updated object states even when the target is temporarily invisible. The model also employs a pathway-selective LoRA to add object control while preserving pretrained camera navigation capabilities, achieving accurate composable control and maintaining object states across extended off-screen intervals.
By Bohai Gu, Taiyi Wu, Yueyang Yuan, Jian Liu, Xiaocheng Lu, Dazhao Du, Jie Zhang, Jinxiang Lai, Shuai Yang, Xiaotong Zhao, Alan Zhao, Song Guo
Sparse-GS2Mesh is a stereo-aware framework that improves 3D surface reconstruction from sparse camera views by combining 3D Gaussian Splatting (3DGS) with stereo matching techniques. The method begins by initializing 3DGS with epipolar depth priors to reduce overfitting, then applies adaptive baseline selection, stereo-matching‑guided fine‑tuning, and 2D/3D co‑regularization to refine the model. Experiments show a 15% performance gain over state‑of‑the‑art methods in low‑overlap scenarios and comparable results in high‑overlap settings.
By Younghyun Noh, Minje Kim, Tae-Kyun Kim
The paper addresses reward hacking that arises when large language models (LLMs) are post‑trained using cheap surrogate judges instead of expensive human feedback. It introduces envelope sampling, a theoretically grounded recalibration technique that uses a small set of ground‑truth annotated outputs to adjust the surrogate judge before further training. Experiments on clinical note generation and a sycophancy task demonstrate that envelope sampling reduces reward hacking compared to traditional base‑model recalibration methods.
By Sanjit Dandapanthula, Shuvom Sadhuka, Samir Khan, Michael Oberst, Aaditya Ramdas, Alexandra Chouldechova
The paper investigates how large language models (LLMs) used for code generation can produce syntactically plausible but incorrect programs, a phenomenon termed overconfident failure. By evaluating four open-source code models across three execution-based benchmarks, the authors examine whether token-level uncertainty metrics can predict execution correctness, analyze overconfidence at both global and local levels, and test mitigation strategies such as selective generation and instruction tuning. Their findings show that existing uncertainty signals are only partially reliable, overconfidence persists across models, instruction tuning can increase certainty without improving correctness discrimination, and common mitigation techniques do not fully resolve the issue.
"whyItMatters":"The study highlights that current uncertainty metrics and mitigation strategies are insufficient to reliably detect or prevent incorrect code generation, underscoring the need for better reliability signals in LLM-based code synthesis."
By Ravishka Rathnasuriya, Wei Yang
Optimal Transport Depth Up-Scaling (OT‑DUS) is a method for expanding large language models by inserting new layers into a pre‑trained model instead of training from scratch. It uses Optimal Transport theory to align and fuse functionally corresponding neurons across adjacent base layers, avoiding the neuron permutation mismatch that plagues existing depth‑up‑scaling techniques. Experiments show OT‑DUS outperforms other methods in both general and specialized domains, and inserting new layers at higher positions improves both performance and training efficiency.
By Mingzi Cao, Xi Wang, Nikolaos Aletras
The paper introduces CAMEO, a framework for generating ultrasound reports from multiple images that emphasizes clinical awareness and evidence grounding. It consists of three stages: learning visual-language primitives, cross-view evidence grounding using distilled image-report pairs, and clinically aware preference alignment with error-oriented pairs. Experiments on the USReport-Distilled benchmark show significant improvements over prior models in BLEU, ROUGE, METEOR, and ClinicalScore metrics.
By Yuchen Yang, Xin Wang, Lufan Wang, Yinghong Pan, Yujuan Feng, Yuqing Yang
The paper introduces Learnable Trust‑Boundary Delimiters (LTBD), a lightweight defense that embeds learnable delimiters into LLM inputs to explicitly mark trusted user instructions versus untrusted external data. LTBD keeps the model’s parameters unchanged, achieving superior performance over inference‑time defenses and matching training‑based methods while adding negligible inference overhead. Experiments show near‑zero attack success rates on AlpacaFarm and TaskTracker, and the method remains robust even against adaptive attackers who know the defense.
By Luman Zhao, Minghui Xu, Yue Zhang, Yijun Yang
The paper investigates why visual instance retrieval often fails when the same object appears at different apparent sizes in query and gallery images. It identifies object-to-image (O2I) ratio mismatch—where the object occupies different fractions of the two images—as the dominant cause of performance degradation, rather than resolution loss. Experiments on a controlled benchmark with 3,021 Objaverse objects show that over 80% of cross-distance degradation is due to O2I mismatch for most pretrained backbones, and that multi-scale architectures only reduce resolution effects but remain vulnerable to O2I mismatch. The authors demonstrate that query-side scale augmentation and an OWLv2 crop reranker achieve state‑of‑the‑art results on ILIAS 100M without retraining the gallery, and that a LoRA fine‑tune can match these gains in a single forward pass, indicating that O2I robustness can be learned.
By Boaz Meivar, Ofir Kedem, Amit Edenzon, Gal Chechik, Shai Avidan
The paper evaluates eight open‑source lightweight Vision‑Language Models (up to 7 B parameters) for converting document images into structured JSON, focusing on heritage collections with historical handwriting and non‑standard layouts. It compares zero‑shot, few‑shot, and fine‑tuned settings, measuring performance with CER, ANLS*, and mAP‑F1, and examines the effects of hyperparameter tuning, image preprocessing, and multi‑stage training. The study demonstrates that adapted VLMs can serve as a sustainable, private alternative to commercial OCR systems for institutional archives.
By Uddipan Basu Bir, Vincent Christlein, Andreas Maier, Mathias Zinnen
CausalDreamer enhances generative world models by freezing the original video tokenizer and re-encoding its latent space into a factored representation that separates controllable versus uncontrollable dynamics and reward-relevant versus reward-irrelevant information. The pretrained dynamics model is fine‑tuned to predict this factored representation, enabling more accurate model‑predictive planning. On the MMBench2 benchmark, CausalDreamer improves normalized scores by 14% on clean tasks and 25% on manipulated variants compared to the baseline, while both models perform poorly on entirely new environments.
By Prince Jha, Nils Lukas, Kun Zhang, Salem Lahlou
The paper introduces AyurParam-2.9B, a bilingual language model fine‑tuned for Ayurveda using a curated dataset of classical texts and clinical guidance in English and Hindi. It outperforms other open‑source instruction‑tuned models of similar size on the BhashaBench‑Ayur benchmark and rivals larger models in performance. The work demonstrates that authentic domain adaptation and high‑quality supervision are essential for reliable, culturally congruent AI in specialized medical fields.
By Sravan Gorugantu, Mohd Nauman, Vijay Devane, Shyam Pawar, Viraj Thakur, Kundeshwar Pundalik, Piyush Sawarkar, Rohit Saluja, Maunendra Desarkar, Ganesh Ramakrishnan
The paper introduces FedRouter, a personalized federated learning framework that clusters tasks rather than clients to build specialized language models. It employs local and global clustering of adapters to associate them with specific tasks and includes an evaluation router that directs test samples to the most suitable adapter. Experiments on a multitask dataset show FedRouter outperforms existing methods, achieving up to 6.1% relative improvement under task interference and 136% under generalization scenarios.
By Gabriel U. Talasso, Meghdad Kurmanji, Allan M. de Souza, Nicholas D. Lane, Leandro A. Villas
The paper evaluates EEG foundation models (EEG‑FMs) beyond clean accuracy, testing six models and a supervised baseline across ten datasets. It finds that no single model excels in all robustness tests, that attribution methods reveal task‑appropriate brain region relevance, and that poor head‑only performance is largely due to pooling strategies rather than representation quality. The study also shows that fine‑tuning repurposes late blocks while early blocks already encode task information.
By Urban \v{S}irca, Maryam Alimardani, Stefanos Zafeiriou, Konstantinos Barmpas