The paper introduces MOVE, a framework for multimodal open‑world verification and expansion in graph learning. MOVE jointly uses visual tokens, textual attributes, and graph context to identify nodes that cannot be assigned to existing classes, then employs a multimodal LLM to generate candidate class descriptions. It selectively expands the class space only when multimodal evidence consistently supports the new classes, avoiding redundancy, and reports an average 11.87% improvement across unknown recognition, open‑domain annotation, and downstream graph learning tasks.
By Zekai Chen, Jiayang Xing, Xun Wu, Miao Zhang, Xunkai Li, Kairui Yang, Zhengyu Wu, Xu Wang, Rong-Hua Li, Guoren Wang
The paper investigates the reliability of attention‑head ablation as a causal inference tool in language models. Using GPT‑2 small, the authors find that a natural post‑projection zeroing method is almost uncorrelated with a corrected pre‑projection ablation and yields a completely different set of top‑5 important heads. They also show that binary accuracy can mask effects near performance floors or ceilings, whereas gold‑token log‑probability provides a graded signal. By employing a discovery/held‑out split and 1,000 matched random‑head and layer‑matched‑head controls, the corrected per‑head effect ranking remains highly stable (Spearman ρ = 0.974) and the top‑5 heads significantly outperform both control distributions (Monte Carlo p = 0.001). However, evidence for task specificity is weak on GPT‑2, and replication on DistilGPT‑2 confirms the intervention‑semantic and matched‑control findings.
By Juli Huang
The paper introduces Density-Aware Reward Aggregation (DARA), a method that adjusts reward weighting in multi-reward reinforcement learning based on the density of active rewards within rollout batches. By applying an inverse-square-root density correction, DARA gives more weight to less frequently active rewards, enabling faster learning of targeted behaviors. Experiments on tool calling and mathematical reasoning demonstrate that DARA achieves comparable final performance while reducing training steps by up to 26% and 65% respectively.
By Tong Zheng, Skylar Zhai, Zhan Cheng, TianMing Sha, Youling Huang, Shuo Zhou, Shaotong Qi, Jingcheng Liang, Xuwei Ding, Pengcheng Xu
The study evaluates whether biological reasoning models actually use their biological inputs by testing six models on DNA, protein, and single‑cell tasks. By perturbing one biological input while keeping others fixed, the authors find that many models (e.g., Evo2, ESM3, BioReason, BioReason‑Pro) rely primarily on textual information, with minimal impact from the biological representations. In contrast, models like ChatNT, Prot2Text‑V2, CellWhisperer, and Cell2Sentence‑Scale show greater dependence on their biological inputs, yet overall accuracy gains do not consistently reflect increased biological input contribution.
By Ada Fang, Nikitha Thoduguli, Lukas Fesser, Hanlin Zhang, Sham M. Kakade, Marinka Zitnik
HADRec is a Hierarchy-Aware Drug Recommendation framework that fuses molecular knowledge and electronic health records to improve medication recommendation. It uses LLaMA-7B to encode clinical notes, ChemBERTa to encode drug SMILES strings, and a cross‑attention mechanism for multimodal fusion, while a hierarchical predictor and consistency constraint loss enforce adherence to the ATC classification system. Experiments on MIMIC‑III and MIMIC‑IV show state‑of‑the‑art performance, strong generalization, and well‑calibrated predictions, with counterfactual evaluation indicating clinically aligned reasoning.
By Junke Wang, Hongshun Ling, Li Zhang, Jinjing Wu, Tong Shao, Fang Wang, Yuan Gao
The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.
By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu
Range-GRPO introduces a semi‑supervised post‑training framework that uses conformally calibrated reward ranges instead of single point scores for large language models. By comparing reward ranges pairwise within rollout groups, the method incorporates reward uncertainty into both the magnitude and direction of learning signals. Experiments show that Range‑GRPO outperforms other semi‑supervised approaches on both in‑distribution and out‑of‑distribution tasks while using fewer training resources.
By Ryunyi Lee, Kangjun Noh, Somin Kim, Heedong Kim, Kyungwoo Song
SkillEvoLean introduces a mutation‑enhanced skill evolution framework for Lean theorem provers, jointly refining a high‑level solving policy and its reference knowledge. The method combines progressive updates from successful and failed proof trajectories with mutation‑based exploration when no complete proof is found, sampling mathematical concepts to generate new skill candidates. Evaluations on MiniF2F, PutnamBench, IMO 2025, and USAMO 2026 show significant proof success improvements over baseline approaches.
By Kuo Zhou, ZiXion Yang, Lu Zhang
The paper investigates the components of group attention used in the Chronos-2 time‑series forecasting model, which attends over variates of a group for both multivariate (MV) and in‑context learning (ICL) forecasting. By editing the attention matrix at inference, the authors separate the head into a V/O pathway that projects a weighted summary of the group and a Q/K pathway that determines the weights. They find that uniform pooling (V/O without Q/K weighting) improves performance in most sensor‑network configurations, while learned weighting helps MV but degrades ICL in many cases; applying uniform weighting in the first block alone benefits all tested ICL settings.
By Michael Fore, James Mason Inder, Mrishika Nair, Praneetha Vaddamanu, Sharlina Keshava
arXiv:2610. 01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)?
By Yohan Chatelain (Krembil Centre for Neuroinformatics, CAMH, Toronto, Canada), Pablo de Oliveira Castro (Universite Paris-Saclay, UVSQ, LI-PaRAD, Versailles, France)
The paper presents a systematic comparison of recurrent neural networks and Transformer models for iterative diffusion MRI tractography, focusing on training strategies, input representations, and hyperparameter tuning. It introduces a generation‑validation phase that provides streamline‑level supervision, enabling the models to achieve the best performance reported on the ISMRM2015 challenge dataset. The study also evaluates the effects of missing bundles, noisy training data, and invalid fibers, and demonstrates applicability to in‑vivo data from the Tractoinferno database.
By Emmanuelle Renauld, Philippe Poulin, Hugo Larochelle, Antoine Th\'eberge, Maxime Descoteaux
The paper introduces query‑conditioned hypernetworks that predict distributions over LoRA weight updates for large language models. By learning a distribution rather than a single point estimate, the method allows sampling multiple adapted models for the same query, improving performance over deterministic hypernetworks and token‑sampling baselines. The study also shows that these learned updates can transfer across different queries, indicating reusable adaptation patterns.
By Azal Ahmad Khan, Keshav Ramji, Tahira Naseem, Ali Anwar, Ram\'on Fernandez Astudillo
The paper introduces LoopCD, a training‑free contrastive decoding framework that improves token selection in Loop‑Transformer models by comparing the final prediction with earlier recurrent passes. LoopCD operates either in logit space (LoopCD‑Logits) with a single extra output pass or in hidden‑state space (LoopCD‑Hidden) with no output overhead. Across multiple looped Transformer families, LoopCD yields significant performance gains—raising pass@1 scores on tasks such as AIME 2024 and HumanEval—while enabling a reduction in the number of recurrent loops and a corresponding decrease in inference FLOPs.
By Weihao Liu, Huangjie Zheng, Tianrong Chen, Rohit Dilip, Richard He Bai, Yizhu Jiao, Yuyang Wang, Ruixiang Zhang
The paper introduces a particle‑level generative model that uses residual‑quantized full‑event data to enable fast, ML‑based surrogate simulation for collider events. It demonstrates conditional generation from detector‑stable particles, explores scaling across dataset and model sizes, and shows that token‑level loss predicts downstream physical fidelity. The work offers an empirical framework for scalable collider full‑event generation using residual‑quantized representations.
By Dan Godi, Dmitrii Kobylianskii, Eilam Gross
AnchorPrompt is an adaptation technique for large audio‑language models that keeps the base model frozen and learns a single block of prompt vectors inserted at the decoder input. By training these prompts through self‑distillation on diverse audio and text perturbations, the method improves answer consistency and reduces hallucinations across multiple benchmarks. The approach is perturbation‑agnostic at inference, enabling zero‑shot transfer to unseen distortions such as reverberation and choice permutations.
By Pooneh Mousavi, Amir Ivry, Mirco Ravanelli, Cem Subakan
The paper introduces ONLINE LLM PICKER, a framework for active model selection of large language models in streaming settings. It selects the most informative prompts for annotation within a limited budget, enabling the identification of the best or near‑best model among many candidates. Experiments on 10 datasets and over 130 language models show up to 71.67% savings in annotation cost and a reduction in regret by up to 2.51× when using the chosen model for sequential generation.
By Alessandro Turrin, Patrik Okanovic, Torsten Hoefler, Nezihe Merve G\"urel
The paper introduces LLMLeak, a covert exfiltration technique that exploits Large‑Language‑Model (LLM) web‑fetching capabilities to transmit confidential data. By embedding a secret into a URL and presenting the corresponding website as part of a legitimate task, the malicious client tricks the LLM into fetching the URL, thereby leaking the secret to an attacker‑controlled server. Experiments on eleven open‑parameter models show a 79.7% success rate, and a real‑world case study confirms the attack’s practicality.
By Alessandro Pegoraro, Daryan Merx, Phillip Rieger, Ahmad-Reza Sadeghi
The paper introduces Universal Byte-Level Encoding (UBE), a dual‑alphabet tokenizer that routes 3‑4‑byte UTF‑8 characters through UTF‑16 while keeping 1‑2‑byte characters on the UTF‑8 path. This design lowers the worst‑case token‑budget disparity for high‑premium scripts without increasing costs for efficient English spans, and it preserves standard BPE merges and exact decoding. In extensive Unicode audits and multilingual language‑model experiments, UBE matches or improves token‑count efficiency and context usability compared to traditional byte‑pair encoding.
By Hyunsik Kim, Youngmoon Jung
SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.
By Jihun Kim, Dongyeop Lee, Namhoon Lee
The paper introduces dFlowGRPO, a reinforcement learning framework tailored for discrete flow models (DFMs). It generalizes previous work on diffusion large language models by supporting various probability paths and non-masked source distributions, and formulates denoising as a Markov decision process that leverages transition rates and posterior models. Experiments on the multimodal DFM FUDOKI show that dFlowGRPO outperforms existing GRPO methods on text‑to‑image generation and matches continuous flow models on multimodal understanding tasks.
By Zhengyan Wan, Yidong Ouyang, Panwen Hu, Qiang Sun