The paper introduces SparLeak, a side‑channel attack that exploits a new GPU micro‑architectural leakage called Sparsity‑Induced Memory Access (SIMA) caused by sparse attention in large language models. By capturing SIMA traces during LLM inference, SparLeak can infer user‑query attributes from prefill‑phase traces and reconstruct autoregressive responses from decoding‑phase traces. Experiments on three LLM architectures, three sparse attention mechanisms, and three privacy‑sensitive datasets show high success rates (90.9% for attribute inference and 87.3% for response reconstruction), underscoring the need to address SIMA leakage in sparse‑attention deployments.
By Fahao Chen, Linkang Du, Jinhao Zhou, Peng Li, Zhou Su
The paper introduces ID Balancing, an Integral‑Derivative controller that improves expert load balance in extremely sparse Mixture‑of‑Experts (MoE) models. By scaling its integral term with load error and activating the derivative term only when imbalance worsens, ID Balancing achieves over 50% reduction in worst‑case backbone MaxVio and 12% reduction in training‑average backbone MinVio compared to leading baselines, while preserving language‑modeling performance across various routing settings. The method’s benefits grow with increased sparsity, making it a promising approach for scaling larger MoE models.
By Peng Jin, Zihan Qiu, Zekun Wang, Bo Zheng, Yang Xu, Tian Xie, Xiao Li, Huaqing Zhang, Haoran Lian, Rui Men, Dayiheng Liu
The paper investigates rounding strategies for low‑precision recurrent state caches in Mamba‑style and hybrid language models. It shows that a deterministic golden‑ratio Weyl dither consistently yields quantized models closer to full precision than stochastic rounding, across various models, storage formats, and long decoding horizons, without extra cost. In contrast, round‑to‑nearest can appear effective in short tests but degrades over long generations due to error accumulation.
By Snigdha Chandan Khilar
The paper investigates test‑time scaling for language models, noting that sequential methods often stall due to entering attractors—sets of answers that block further exploration. Across 27 experiments, over half of sequential trajectories hit an attractor within four iterations. Introducing a simple model‑mixing intervention reduces attractor incidence by 21.2 percentage points, expands solution coverage beyond a compute‑matched parallel baseline, and boosts recursive self‑aggregation accuracy by at least 2.2 percentage points.
By Joseph Rance, Fabio Pizzati, Juil Sock, Woody Bayliss, Marc G\'orriz Blanch, Philip Torr, Adel Bibi
The paper argues that the policy governing how tensors are rounded and reused during the backward pass—termed the backward‑state policy—is an integral part of the learning algorithm, not merely a memory or precision detail. Experiments with 390 M‑parameter models show that whether the backward pass reuses a forward’s rounded output or generates a new rounding can decisively affect training success, even when copy accuracy is high. The authors propose a method to determine, for each operator, which value should be read or substituted to preserve the correct gradient, and validate these predictions on PyTorch and Transformer Engine.
By Shuxiao Xie, Shuyang Xie, Dezhi Ran, Wei Yang, Tao Xie
The paper evaluates how language models of different sizes self-assess their confidence on question‑answering tasks across general and specialized domains. It finds that while accuracy drops for smaller models and more specialized tasks, the reliability of self‑evaluated confidence signals remains largely stable. Consequently, even less capable models can provide reasonable confidence estimates, making them suitable for resource‑constrained applications.
By Idil Kapikiran, Thomas Decker, Thomas Runkler
The paper introduces Fork-dLLM, a hybrid sampling strategy for diffusion language models that combines parallel token generation with autoregressive ordering only at uncertain fallback steps, thereby avoiding the flexibility trap that reduces generation diversity. It also presents ForkGRPO, an extension that applies the GRPO objective selectively during post‑training rollouts, preserving policy‑likelihood ratios while cutting rollout and optimization costs. Experiments show Fork-dLLM achieves pass@k scaling comparable to autoregressive sampling but 2–3× more efficient, and ForkGRPO matches or surpasses AR‑based GRPO baselines at significantly lower training expense.
By Stipe Frkovi\'c, Metod Jazbec, Christian A. Naesseth
The paper introduces a preemptive unlearning technique called gradient sealing to prevent large language models (LLMs) from acquiring forbidden capabilities during downstream fine‑tuning. It shows that existing retrospective unlearning methods fail to block hidden gradient pathways that can later enable illicit knowledge. By pushing relevant pre‑activations into the negative region, gradient sealing effectively blocks these pathways and demonstrates stronger resistance to downstream acquisition across multiple LLM families.
By Kemou Li, Qizhou Wang, Yue Wang, Fengpeng Li, Zhuan Shi, Negar Rostamzadeh, Golnoosh Farnadi, Masashi Sugiyama, Jiantao Zhou
arXiv:2609. 39892v1 Announce Type: new Abstract: Looped Transformers repeatedly apply the same set of Transformer layers, giving them a recurrent architecture for latent computation.
By Jiaju Wu, Yi Hu, Muhan Zhang
arXiv:2609. 40148v1 Announce Type: new Abstract: Power-law learning curves are often treated as fixed properties of a model and its data, although learning-rate and batch-size schedules can change the observed loss.
By Yichen Wang, Fanghui Liu, Yudong Chen
The study shows that previously reported gains in decoding spoken words from non‑invasive brain recordings can largely be replicated without any brain data, due to a timing shortcut where overlapping windows reveal word durations. By processing each window independently rather than jointly, the authors eliminate this shortcut, enabling the neural network to learn genuine brain‑based word information. This change improves performance and makes two existing strategies—aggregating predictions from multiple neural responses and incorporating a pretrained language model—significantly more effective, achieving a word error rate of 36.6% on a perceived speech benchmark.
By Dulhan Jayalath, Oiwi Parker Jones
The paper introduces VirusCascade, a black‑box attack that exploits the collaborative reflection mechanism in LLM‑powered recommender agents. By leveraging reflective persistence and cross‑agent propagation, the attack promotes a target item through semantic rationalisation and structural positioning, achieving state‑of‑the‑art targeted exposure on four real‑world datasets. The study demonstrates that existing static‑pipeline attacks cannot exploit this recurrent, multi‑agent amplification pathway.
By Yurong Hao, Wen Zhou, Guowei Guan, Tiantong Wu, Fuyao Zhang, Wei Yang Bryan Lim
The paper introduces the Posterior Concentration Phenomenon (PCP), a length‑dependent failure mode where probability‑based rewards collapse to a narrow interval for long reasoning traces, destabilizing verifier‑free reinforcement learning. To address this, the authors propose RLCPR, a framework that uses uncertainty‑aware data sampling and concentration‑aware regularization to mitigate PCP, improving token efficiency and outperforming state‑of‑the‑art baselines on multiple reasoning benchmarks.
By Shiu-Hong Kao, Yubo Zhao, Zhenyu Tian, Pengzhan Sun, Yicong Li, Angela Yao
The paper presents a study on securing large language model (LLM) inference services against adversarial use, such as jailbreaking, denial‑of‑service, and distillation attacks. It introduces a structural causal model to generate a realistic, labeled dataset of user sessions, including coordinated multi‑account campaigns and varying label observability. Using this dataset, the authors train a gradient‑boosted detector that achieves near‑perfect binary classification on oracle labels but shows lower performance on operational labels, and they demonstrate a simple thresholded engine that improves attack‑type attribution accuracy.
By Keifer Lee
ThunderEP is a new communication design for expert parallelism on PCIe-connected consumer GPUs that eliminates relay hops, uses DMA engines to avoid GPU compute contention, and reduces CPU polling overhead. Integrated into vLLM, it outperforms NCCL on RTX 4090 and RTX 5090 GPUs, delivering average speedups of 2.00× for dispatch, 1.53× for combine, and up to 1.66× end‑to‑end over existing MoE inference frameworks.
By Jaehwan Lee, Sangmin Lee, Chaewon Kim, Junsik Shin, Jaejin Lee
The article outlines a Ph.D. research agenda aimed at creating provably efficient and practical algorithms for data‑driven sequential decision‑making under uncertainty. It focuses on reinforcement learning and multi‑armed bandits, targeting applications such as recommendation systems, computer networks, video analytics, and large language models. The work seeks to overcome limitations of existing methods—such as reliance on idealized models, lack of robustness to adversarial perturbations, and poor instance‑dependent performance—by developing algorithms that are more efficient, robust, instance‑adaptive, and generalizable to new environments.
By Zhiyong Wang
The paper introduces SEAR, a four-task benchmark that evaluates audio language models (ALMs) for audio deepfake detection by requiring acoustic evidence identification, quantification, deepfake detection, and forensic rationale generation. It also proposes BAEA, an acoustic evidence agent that equips a frozen ALM with controlled acoustic tools under fixed or adaptive evidence-acquisition policies. Experiments with six ALMs show a gap between plausible rationales and verifiable acoustic evidence reasoning, and demonstrate that BAEA‑Fixed improves final verdicts and forensic rationales while revealing that misleading evidence harms detection and grounding performance.
By Rong Wan, Suliu Qin, Jiaxi Li, Wei Xie, Wenwu Wang, Xiaolong Han, Lu Yin, Xilu Wang
The paper introduces Model-to-Data (M2D) distillation, a method that transfers properties learned by a complex graph neural network (GNN) teacher into the graph data itself. By jointly learning augmented node features and graph structure, M2D encodes the teacher’s behavior, allowing simpler GNNs to recover high predictive performance, fairness, and robustness. The resulting graph can be reused with various downstream models, enabling them to approximate sophisticated teachers such as fairness-aware GNNs, Graph Attention Networks, and Graph Transformers.
By Debolina Halder Lina, Arlei Silva
GPEC is a lightweight pre‑LLM Gaussian Process Embedding Correction that refines visual representations for cardiac ultrasound caption generation. By inserting a residual correction layer between the visual projection and the language model, it aligns projected embeddings with annotation‑guided targets derived from structured video annotations. Experiments show that GPEC improves caption similarity and content alignment while adding less than 0.05 s of inference overhead, all without fine‑tuning the pretrained multimodal backbone.
By Arefeh Rezaei
The paper introduces RateAR, a benchmark of AR images and videos that captures perceptual factors such as object placement, scale, and shadow consistency. Eleven commercial vision‑language models (VLMs) are evaluated on this benchmark, with VLM‑based quality predictions showing strong correlation with human judgments (Spearman’s ρ up to 0.8695). Using contextual prompting, the authors build an automated AR content adjustment system that, in a 21‑participant study, improved placement and size coherence of virtual content for over 90% of users.
By Elias Rotondo (Duke University), Lin Duan (Duke University), Yanming Xiu (Duke University), Sangjun Eom (Duke University), Conrad Li (Duke University), Maria Gorlatova (Duke University)