HADRec is a Hierarchy-Aware Drug Recommendation framework that fuses molecular knowledge and electronic health records to improve medication recommendation. It uses LLaMA-7B to encode clinical notes, ChemBERTa to encode drug SMILES strings, and a cross‑attention mechanism for multimodal fusion, while a hierarchical predictor and consistency constraint loss enforce adherence to the ATC classification system. Experiments on MIMIC‑III and MIMIC‑IV show state‑of‑the‑art performance, strong generalization, and well‑calibrated predictions, with counterfactual evaluation indicating clinically aligned reasoning.
By Junke Wang, Hongshun Ling, Li Zhang, Jinjing Wu, Tong Shao, Fang Wang, Yuan Gao
The paper investigates how test‑time computation can enhance language models and at what cost, introducing the SELF‑POT benchmark to evaluate this across competition mathematics, competitive programming, and agentic workflows. SELF‑POT separates candidate coverage from final accuracy, tracks correctness transitions under revision, and measures protocol completion alongside task success. Using a unified budget rule, the study compares direct inference, parallel sampling, and self‑revision across five low‑cost reasoning models, revealing that selection rules and failure handling significantly influence gains and cost savings.
By Bangji Yang, Jingyuan Li, Jiajun Fan, Yi Evie Zhang, Ruihan Guo, Hongba Ma, Neil He, Chumeng Liang, Qinglong Zheng, Zhanghan Ni, Ge Liu
Range-GRPO introduces a semi‑supervised post‑training framework that uses conformally calibrated reward ranges instead of single point scores for large language models. By comparing reward ranges pairwise within rollout groups, the method incorporates reward uncertainty into both the magnitude and direction of learning signals. Experiments show that Range‑GRPO outperforms other semi‑supervised approaches on both in‑distribution and out‑of‑distribution tasks while using fewer training resources.
By Ryunyi Lee, Kangjun Noh, Somin Kim, Heedong Kim, Kyungwoo Song
SkillEvoLean introduces a mutation‑enhanced skill evolution framework for Lean theorem provers, jointly refining a high‑level solving policy and its reference knowledge. The method combines progressive updates from successful and failed proof trajectories with mutation‑based exploration when no complete proof is found, sampling mathematical concepts to generate new skill candidates. Evaluations on MiniF2F, PutnamBench, IMO 2025, and USAMO 2026 show significant proof success improvements over baseline approaches.
By Kuo Zhou, ZiXion Yang, Lu Zhang
The paper investigates the components of group attention used in the Chronos-2 time‑series forecasting model, which attends over variates of a group for both multivariate (MV) and in‑context learning (ICL) forecasting. By editing the attention matrix at inference, the authors separate the head into a V/O pathway that projects a weighted summary of the group and a Q/K pathway that determines the weights. They find that uniform pooling (V/O without Q/K weighting) improves performance in most sensor‑network configurations, while learned weighting helps MV but degrades ICL in many cases; applying uniform weighting in the first block alone benefits all tested ICL settings.
By Michael Fore, James Mason Inder, Mrishika Nair, Praneetha Vaddamanu, Sharlina Keshava
arXiv:2610. 01889v1 Announce Type: new Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)?
By Yohan Chatelain (Krembil Centre for Neuroinformatics, CAMH, Toronto, Canada), Pablo de Oliveira Castro (Universite Paris-Saclay, UVSQ, LI-PaRAD, Versailles, France)
The paper presents a systematic comparison of recurrent neural networks and Transformer models for iterative diffusion MRI tractography, focusing on training strategies, input representations, and hyperparameter tuning. It introduces a generation‑validation phase that provides streamline‑level supervision, enabling the models to achieve the best performance reported on the ISMRM2015 challenge dataset. The study also evaluates the effects of missing bundles, noisy training data, and invalid fibers, and demonstrates applicability to in‑vivo data from the Tractoinferno database.
By Emmanuelle Renauld, Philippe Poulin, Hugo Larochelle, Antoine Th\'eberge, Maxime Descoteaux
The paper introduces query‑conditioned hypernetworks that predict distributions over LoRA weight updates for large language models. By learning a distribution rather than a single point estimate, the method allows sampling multiple adapted models for the same query, improving performance over deterministic hypernetworks and token‑sampling baselines. The study also shows that these learned updates can transfer across different queries, indicating reusable adaptation patterns.
By Azal Ahmad Khan, Keshav Ramji, Tahira Naseem, Ali Anwar, Ram\'on Fernandez Astudillo
The paper introduces LoopCD, a training‑free contrastive decoding framework that improves token selection in Loop‑Transformer models by comparing the final prediction with earlier recurrent passes. LoopCD operates either in logit space (LoopCD‑Logits) with a single extra output pass or in hidden‑state space (LoopCD‑Hidden) with no output overhead. Across multiple looped Transformer families, LoopCD yields significant performance gains—raising pass@1 scores on tasks such as AIME 2024 and HumanEval—while enabling a reduction in the number of recurrent loops and a corresponding decrease in inference FLOPs.
By Weihao Liu, Huangjie Zheng, Tianrong Chen, Rohit Dilip, Richard He Bai, Yizhu Jiao, Yuyang Wang, Ruixiang Zhang
The paper introduces a particle‑level generative model that uses residual‑quantized full‑event data to enable fast, ML‑based surrogate simulation for collider events. It demonstrates conditional generation from detector‑stable particles, explores scaling across dataset and model sizes, and shows that token‑level loss predicts downstream physical fidelity. The work offers an empirical framework for scalable collider full‑event generation using residual‑quantized representations.
By Dan Godi, Dmitrii Kobylianskii, Eilam Gross
AnchorPrompt is an adaptation technique for large audio‑language models that keeps the base model frozen and learns a single block of prompt vectors inserted at the decoder input. By training these prompts through self‑distillation on diverse audio and text perturbations, the method improves answer consistency and reduces hallucinations across multiple benchmarks. The approach is perturbation‑agnostic at inference, enabling zero‑shot transfer to unseen distortions such as reverberation and choice permutations.
By Pooneh Mousavi, Amir Ivry, Mirco Ravanelli, Cem Subakan
The paper introduces ONLINE LLM PICKER, a framework for active model selection of large language models in streaming settings. It selects the most informative prompts for annotation within a limited budget, enabling the identification of the best or near‑best model among many candidates. Experiments on 10 datasets and over 130 language models show up to 71.67% savings in annotation cost and a reduction in regret by up to 2.51× when using the chosen model for sequential generation.
By Alessandro Turrin, Patrik Okanovic, Torsten Hoefler, Nezihe Merve G\"urel
The paper introduces LLMLeak, a covert exfiltration technique that exploits Large‑Language‑Model (LLM) web‑fetching capabilities to transmit confidential data. By embedding a secret into a URL and presenting the corresponding website as part of a legitimate task, the malicious client tricks the LLM into fetching the URL, thereby leaking the secret to an attacker‑controlled server. Experiments on eleven open‑parameter models show a 79.7% success rate, and a real‑world case study confirms the attack’s practicality.
By Alessandro Pegoraro, Daryan Merx, Phillip Rieger, Ahmad-Reza Sadeghi
The paper introduces Universal Byte-Level Encoding (UBE), a dual‑alphabet tokenizer that routes 3‑4‑byte UTF‑8 characters through UTF‑16 while keeping 1‑2‑byte characters on the UTF‑8 path. This design lowers the worst‑case token‑budget disparity for high‑premium scripts without increasing costs for efficient English spans, and it preserves standard BPE merges and exact decoding. In extensive Unicode audits and multilingual language‑model experiments, UBE matches or improves token‑count efficiency and context usability compared to traditional byte‑pair encoding.
By Hyunsik Kim, Youngmoon Jung
SeedFlood is a novel decentralized fine‑tuning method for large language models that scales to billions of parameters and hundreds of clients. It leverages the seed‑reconstructible structure of zeroth‑order gradients to reduce message sizes to near‑zero, enabling efficient flooding across the network. Experiments show SeedFlood outperforms standard zeroth‑order baselines in communication efficiency and generalization, and rivals first‑order gossip methods while incurring far less communication cost.
By Jihun Kim, Dongyeop Lee, Namhoon Lee
The paper introduces dFlowGRPO, a reinforcement learning framework tailored for discrete flow models (DFMs). It generalizes previous work on diffusion large language models by supporting various probability paths and non-masked source distributions, and formulates denoising as a Markov decision process that leverages transition rates and posterior models. Experiments on the multimodal DFM FUDOKI show that dFlowGRPO outperforms existing GRPO methods on text‑to‑image generation and matches continuous flow models on multimodal understanding tasks.
By Zhengyan Wan, Yidong Ouyang, Panwen Hu, Qiang Sun
The paper introduces TANGO, a Token‑Aggregated Nonlinear Gating Operator that blends cross‑token mixing and token‑wise transformation in transformer architectures. By computing a nonlinear gate per source token and averaging these gates for each destination, TANGO forms a source‑conditioned linear operator that improves predictive performance. Experiments on web text, formal mathematics, and code show that full‑prefix TANGO achieves the lowest test negative log‑likelihood across 16 settings, while a narrower variant offers substantial throughput gains with only a modest increase in loss.
By Joshua Nunley
LumoTree is a verifier designed for hybrid language models that ensures a single coherent continuation across recurrent, convolution, and attention states. It executes recurrent paths in parallel, reuses state tiles, and coordinates replay, history gathering, and cache remapping through a shared logical tree. Experiments demonstrate exact candidate‑selection parity and recurrent agreement within error bounds, and a Qwen3.8‑27B deployment achieved 25.63 pooled tokens/s on a single NVIDIA DGX Spark.
By Zhiyuan Ma
Attention Manifolds introduce learned 2D B-spline surfaces that modulate each value dimension in transformer attention based on query-key interactions. Applied to LLaMA models, they reduce WikiText-2 perplexity by 2–2.5 points with minimal parameter overhead and alter greedy-decoded outputs in 69–83% of diverse prompts, especially on ambiguous inputs. The surfaces can be edited mechanically to steer or block value flow, offering a geometric tool for improving output quality and potentially enhancing safety.
By Naveen Mysore
Large Language Bayes (LLB) samples probabilistic programs from a language model, runs approximate inference on each, and averages them weighted by an exponentiated evidence bound. The authors demonstrate that this weighting is not invariant to reparameterisation, unlike the log marginal likelihood, leading to significant discrepancies in weights across different program formulations. These discrepancies can reach up to 31.9×, affect Bayes factors, and introduce controlled errors in posterior estimates.
By Jian Xu