Hybrid language models combine attention with a fixed-size recurrent state, yet the distinct roles of each component are not well understood. The authors introduce two cache-level interventions—split-prefill and state-swap—to isolate the contributions of the KV cache (attention) and the recurrent state. Experiments on Qwen3.5 and Falcon-H1 show that exact retrieval depends almost entirely on attention, while output language and persona rely mainly on recurrence, with the state-swap intervention confirming that answers derive from the KV side and language from the recurrent side.
By Kirill Afendulev, Alexey Dontsov, Elena Tutubalina, Anton Korznikov
The paper investigates regression with bounded responses, comparing two learning frameworks: model selection aggregation, which requires improper algorithms to achieve minimax excess risk, and universal learning, where empirical risk minimization suffices for exponential learning rates. For finite hypothesis classes, the authors show that the $Q$-aggregation estimator simultaneously attains minimax optimal tails and exponential universal rates, while other common estimators fail to do so. For countably infinite classes, they prove an inherent trade‑off between exponential universal and minimax uniform rates, resolved by combining optimal algorithms from each framework via $Q$-aggregation.
By Mikael M{\o}ller H{\o}gsgaard, Patrick Rebeschini, Tobias Wegel
Scale-QLoRA introduces a method for merging LoRA adapters into native 4‑bit microscaling language models without altering the quantized code plane. By training only the per‑block scale field and freezing the E2M1 code, the merged model remains code‑invariant and can be deployed as a single checkpoint. Experiments on four models and tasks show that Scale‑QLoRA achieves accuracy‑lossless merging comparable to merge‑aware QAT‑LoRA, while offering benefits such as exact rollback, code‑plane deduplication, and a 125× faster scale‑only task swap.
By Tung-Ling Li, Jiale Huang, Lee-Chi Wang, Janaki Ram Gotei
The paper presents a unified compression framework for Vision Transformers aimed at on‑device plant disease detection in resource‑constrained agricultural settings. It combines Hessian‑Balanced Adaptive Block Pruning, quantization, and attention‑based knowledge distillation, evaluating each component separately before integrating the best performers into a deployment pipeline. On a chilli disease dataset, the compressed models achieve accuracy comparable to the FP32 baseline while reducing model size by 74‑98 %, and the full pipeline attains a 54.5× size reduction to 6.01 MB with 95.13 % accuracy.
By Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith, Gangireddy Rahul Jogi, Sudheesh Manalil, Arnab Raha, Amitava Mukherjee, Parthasarathy Seethapathy, G. Gopakumar
The paper introduces AdaGate-DF, an adaptive gated deepfake detection framework designed for low-resolution and resource-constrained environments. By leveraging image-quality cues, the system routes samples through a dual multi-exit architecture, allowing high-quality images to exit earlier and reduce computational load. Evaluations on Celeb-DF and FaceForensics++ show that AdaGate-DF outperforms existing models such as MaD-CoRN and DefakeHop++ while maintaining low inference latency and robust performance across varying resolutions and class imbalance scenarios.
By Vaishnavi Sen, Cody Laurie, Rashida Hasan
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
ReCAST is a framework that trains a small student model to classify obfuscated Chinese SMS messages by first learning to detect obfuscated spans, predict obfuscation types, and restore the original text using knowledge distilled from a large teacher model. The student model then performs risk classification, achieving better performance than directly trained baselines on a real-world Chinese SMS benchmark. This approach demonstrates that restoration-aware distillation can produce robust, production‑ready SMS risk classifiers with reduced model size.
By Jieyun Huang, Yi Shen, Kaikai Zhao, Jiangze Yan, Wenjing Zhang, Ping Chen, Ning Wang, Zhaoxiang Liu, Kai Wang, Shiguo Lian
The study explores non‑smooth activation functions—chaotic, stochastic, and fractal—in echo state networks, testing 36,610 reservoir configurations. It finds that functions like the Cantor function preserve the Echo State Property (ESP) even at spectral radii up to 10, outperforming traditional smooth activations in convergence speed. The authors introduce a theoretical framework for quantized activations, defining a Degenerate Echo State Property (d‑ESP) that implies the traditional ESP, and show that preprocessing topology, rather than continuity, governs stability.
By Rae Chipera, Jenny Du, Irene Tsapara
MCPO introduces a two-stage method to compress multimodal chain-of-thought reasoning. The first stage uses a step-level Normalized Cross-Modal Mutual Information pruning algorithm to remove visual-independent steps, reducing redundancy and hallucinations. The second stage fine-tunes the model with an asymmetric multimodal length-controlled preference loss, achieving up to 69.5% shorter reasoning chains and a 3.34× inference speedup while keeping accuracy.
By Guangheng Yang, Zhenliang Ni, Zhenkai Wu, Han Shu, Juan Feng, Wenming Yang, Jie Hu
The paper investigates how model compression impacts test-time adaptation (TTA) in deep neural networks. Using ResNet‑18 and ViT‑Base on CIFAR‑10‑C and ImageNet‑C, the authors evaluate several compression techniques alongside standard TTA methods, introducing a diagnostic framework to assess representational expressivity and adaptation subspace compatibility. Results show that while compressed models maintain high accuracy under supervised adaptation, their TTA performance deteriorates with increased compression due to reduced representational diversity and structural constraints that limit recoverability.
By Francesco Corti, Dong Wang, Young D. Kwon, Cecilia Mascolo, Olga Saukh
The paper introduces the concept of a free pause token, which allows a language model to perform additional computation for each next-token prediction without increasing the sequence length or KV cache usage. This pause token operates as a parallel prediction stream over a shared backbone, improving next-token accuracy by 2–3 centi‑nats on a 1‑billion‑parameter model. While it adds minimal inference overhead, the primary cost lies in training, where compute requirements rise modestly (up to 1.14×) yet most benefits are retained.
By John Langford, Nathan Godey, Giovanni Monea, Yoav Artzi, Harry Dong, Ying Fan, Gustavo de Rosa, Zheng Zhan
CoMAP introduces a framework that jointly evolves textual world models and agent policies through a closed‑loop interaction. At each decision step the world model forecasts future state feedback for candidate actions, while the agent reflects on the reliability of this feedback to refine its action. The resulting on‑policy trajectories are used to self‑distill and update the world model, improving prediction accuracy and long‑horizon decision‑making across embodied planning, web navigation, and tool‑use benchmarks.
By Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li
The paper presents a consensus framework for Bayesian Network fusion that balances dependency preservation with computational tractability by enforcing treewidth constraints. It introduces genetic algorithms featuring advanced initialization, specialized operators, and a tailored fitness function to prioritize shared structures among input networks. Experiments on synthetic and real-world BNs demonstrate that these genetic algorithms outperform adapted methods and greedy baselines.
By Pablo Torrijos, Jos\'e A. G\'amez, Jos\'e M. Puerta, Juan A. Aledo
GrowPage is an on‑demand key–value (KV) budgeting framework designed to improve large language model (LLM) reasoning serving. It treats KV capacity as a runtime resource, using lightweight dual‑timescale query summaries to track recent and long‑term attention patterns and estimate demand evolution. At each capacity boundary, GrowPage either compresses KV states within the current allocation or acquires an additional physical page, integrating with PagedAttention’s page‑level memory abstraction to maintain continuous batching and prefix caching.
By Qiankun Ma, Yanjiang Zhou, Zinan Xiong, Haofei Wang, Zhen Song, Yang Xiang, Ziyao Zhang, Hairong Zheng
SV-WAM is a surround‑view world‑action model that keeps all six camera views while enabling efficient inference by discarding the video branch at deployment. It uses future‑video prediction as dense training supervision and an action‑centered causal mask to prevent action tokens from attending to future‑video tokens during joint denoising. A differentiable drivable‑area compliance regularizer penalizes vehicle‑footprint corners near or crossing drivable boundaries, improving safety and boundary awareness. Experiments on NAVSIMv2 and nuScenes show state‑of‑the‑art planning performance with low latency and strong zero‑shot transfer.
By Jinyang Wang, Shiwei Li, Junjian Wang, Zhiqiang Deng, Jianbin Gao, Yihang Zhao, Liu Liu, Yongjia Zhao, Jinlong Chen, Huirui Xu, Yifeng Pan, Kangwei Liu, Fan Ren, Ji Tao, Minghao Yang
The paper introduces a physics‑informed graph attention network that directly operates on the tetrahedral mesh used in TCAD simulations of FinFET devices. By predicting electrostatic potential and quasi‑Fermi levels at every mesh node and training with both data loss and finite‑volume current‑continuity residuals, the surrogate retains the underlying carrier‑transport physics while achieving size generalization. Benchmarks against Sentaurus Device show sub‑volt RMSE for the drift‑diffusion fields and a per‑design throughput that is orders of magnitude faster, enabling rapid Pareto‑front exploration of large multi‑fin arrays that would otherwise be prohibitively slow to simulate.
By Leonid Popryho, Ayoub Sadeghi, Inna Partin-Vaisband
arXiv:2609. 03378v1 Announce Type: new Abstract: Depth can resolve appearance ambiguity in RGB-D salient object detection (SOD), yet sensor depth is not uniformly reliable.
By Xuehao Wang, Jiaxin Hua, Runmei Li, Zhenyu Wu, Chenglizhao Chen, Ke Gu, Aimin Hao
FlashRender is a few-step generative rendering framework that quickly retakes a source video along a target camera trajectory. It addresses discretization error by introducing Representation Transformation and Alignment (RETA) to align source-video representations with target-video features, reducing denoising trajectory curvature. The model is further refined with a MeanFlow objective and on-policy flow map distillation, achieving video quality and geometric consistency comparable to multi-step baselines at a 25× lower sampling cost while improving camera controllability.
By Byeongjun Park, Byung-Hoon Kim, Hyungjin Chung
EmoDistill is an offline framework that distills emotional negotiation skills from large language model interactions into smaller agents. It separates emotion selection, handled by an Implicit Q‑Learning selector, from emotion‑conditioned expression, learned by a LoRA‑adapted 7B policy via supervised fine‑tuning and judge policy optimization. Experiments across four negotiation domains show that the full EmoDistill policy outperforms vanilla and IQL‑only baselines, while removing the explicit emotion channel markedly reduces negotiation utility and reveals partial, domain‑dependent transfer to unseen counterparties.
By Yunbo Long, Haolang Zhao, Lukas Beckenbauer, Liming Xu, Alexandra Brintrup
The paper introduces Modality‑Specific Frequency Distillation (MSFD), a continual learning framework for video deepfake detection that separates spatial, temporal, and spatiotemporal features in the frequency domain. By preserving each modality independently and applying a cross‑modality decorrelation loss, MSFD adapts to new forgery patterns while maintaining performance across diverse continual deepfake video scenarios. Experiments demonstrate that this approach outperforms state‑of‑the‑art methods in both adaptation and retention.
By Taehoon Kim, Jongwook Choi, Heejae Jo, Byungmin Park, Jongwon Choi