SOD: Step-wise On-policy Distillation for Small Language Model Agents proposes a new framework that adaptively reweights distillation strength at each reasoning step based on step-level divergence. This approach mitigates cascading errors in tool-integrated reasoning by attenuating misleading teacher signals in high-divergence regions while preserving dense guidance where student and teacher align. Experiments on math, science, and code benchmarks show up to 20.86% improvement over the second-best baseline, with a 0.6B student scoring 26.13% on AIME 2025.
By Qiyong Zhong, Mao Zheng, Mingyang Song, Xin Lin, Jie Sun, Houcheng Jiang, Xiang Wang, Junfeng Fang
The paper proposes Self‑Distillation Fine‑Tuning (SDFT) as a method to recover performance in Large Language Models that has been degraded by catastrophic forgetting, quantization, or pruning. It shows that SDFT restores model capabilities by aligning the high‑dimensional manifold of the student model’s hidden layers with that of a teacher model, as measured by Centered Kernel Alignment (CKA). The authors provide both empirical evidence of strong correlation between manifold alignment and performance recovery and a theoretical explanation linking generative capability to the structure of these manifolds.
By Chi Liu, Xin Chen, Xu Zhou, Fangbo Tu, Srinivasan Manoharan
The study evaluates crop‑yield forecasting methods for the 2012 Midwestern US drought, comparing non‑deep learning machine learning models with a deep learning model (VITA) using 16 meteorological predictors. It highlights challenges such as distributional dissimilarity between training and test data, spatial and temporal sparsity, and demonstrates that sample weighting and feature selection improve non‑deep learning models but not VITA. The work contrasts deep versus non‑deep learning approaches and shows how modifications can mitigate issues arising from extreme drought conditions.
By Shrey Gupta, Yi Ming, George Mohler
The paper presents RuntimeGuard‑AI, a prototype that links each deterministic AI policy decision to its source code, writes a privacy‑minimizing record at a chosen synchronization point, and returns an Ed25519‑signed receipt indicating whether the write succeeded. After a crash, the system validates the integrity of records, manifests, shard placement, sequence continuity, and replay identity, while an independent attestation path chains committed records into signed Merkle epochs for auditor verification. Performance results on an Apple M4 Pro show high throughput (up to 27,193 requests/s) with low latency when buffering, but throughput drops and latency rises when per‑record data and full synchronization are used, illustrating a clear durability‑latency trade‑off.
By Neeraj Kumar Singh Beshane
The paper introduces an online generalized-sparsity-constrained regression framework that addresses key challenges in online sparse regression, such as dynamic regularization, memory usage, and real-time computation. It proposes an efficient online hard‑thresholding algorithm that performs closed‑form updates using only summary statistics, achieving global convergence at optimal statistical rates when the projection set is overparameterized. Numerical experiments show the method consistently outperforms existing alternatives in online cardinality‑constrained linear regression and low‑rank matrix sensing.
By Shuoguang Yang, Qiang Sun
The paper investigates how adjusting the sharpness of t‑SNE’s affinity matrix influences neighborhood preservation across scales. By applying a row‑wise power transform parameterized by γ, the authors can smooth or sharpen each row while keeping sparsity and rank order intact, effectively rescaling the Gaussian bandwidth and altering local perplexities. Experiments show that sharpening enhances the retention of the very nearest neighbors, whereas smoothing improves the preservation of broader local neighborhoods, outperforming existing multiscale affinity methods in the mid‑local range.
By Shirin Mohebi, Guillaume Bied, Jefrey Lijffijt
The paper introduces a small language model (SLM)-based framework that detects and classifies GNSS spoofing attacks on autonomous vehicles by converting vehicle states from GNSS and other sensors into structured semantic narratives. The SLM achieves performance comparable to large language models, with an average accuracy of 96.99%, while offering lower inference latency and reduced GPU memory usage. Field tests in Clemson, South Carolina, confirm the framework’s real‑time detection capabilities on resource‑constrained vehicular platforms.
By Abyad Enan, Sagar Dasgupta, Mizanur Rahman, Mashrur Chowdhury
arXiv:2608. 16947v1 Announce Type: cross Abstract: Huang, Lou, and Xiao introduced Dynamic Mixture-of-Experts Serving and gave an O(sqrt(log k))-competitive randomized algorithm for its integral primal problem, where k is the number of replica GPUs beyond the mandatory copy of each expert.
By Ian D'Ambrosio (Nth Research Collective)
The paper explores energy-aware knowledge distillation for large language models (LLMs) used in software engineering tasks such as clone detection, vulnerability prediction, and code summarization. It shows that the commonly used FLOPs metric does not reliably reflect actual energy consumption, and that using energy-surrogate models during distillation can reduce inference energy by up to 90% and memory usage by 86% with only modest accuracy loss. The study demonstrates that guiding distillation with direct energy estimates improves the sustainability and deployability of LLMs on consumer hardware.
By Enrique Barba Roque, Lu\'is Cruz, Annibale Panichella
LoopVLA introduces a recurrent Vision‑Language‑Action architecture that learns to refine multimodal representations, predict actions, and estimate when further refinement is unnecessary. By iteratively applying a shared Transformer block and producing a sufficiency score at each step, it decouples refinement from fixed layer indices and aligns confidence scores with action quality through a self‑supervised objective. Experiments on LIBERO, LIBERO‑Plus, and VLA‑Arena demonstrate that LoopVLA reduces model parameters by 45% and boosts inference throughput up to 1.7× while matching or surpassing strong baselines in task success.
By Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang
arXiv:2608. 17802v1 Announce Type: new Abstract: Let $\varepsilon_1,\ldots,\varepsilon_n$ be independent Rademacher signs and let $a=(a_1,\ldots,a_n)\in\R^n$ satisfy the normalization below.
By Peigan Gao, Jian Qian
ManiCM is a real‑time 3D diffusion policy for robotic manipulation that uses a consistency constraint to enable one‑step inference. The model conditions on point‑cloud input and directly predicts robot actions through a consistency distillation technique, avoiding the need to predict noise. Evaluated on 31 tasks from Adroit and Metaworld, ManiCM achieves an average ten‑fold speedup over state‑of‑the‑art methods while maintaining competitive success rates.
By Zifeng Gao, Guanxing Lu, Tianxing Chen, Wenxun Dai, Ziwei Wang, Chao Shang, Wenbo Ding, Yansong Tang
KernelArc is a multi-agent framework designed to autonomously optimize GPU kernels across diverse workloads. It employs strategy-specialized agents that run concurrently, coordinating via conclusions-only shared memory, a deterministic benchmark guard, and read-only cross-agent state with plateau-triggered drafting. Evaluated on NVIDIA H100 and B200 GPUs with SOL-ExecBench workloads, KernelArc produced top-ranked implementations for tasks such as BF16 GEMM, cuBLASLt configuration tables, and various attention mechanisms, achieving first place on several leaderboard categories.
By Joyjit Kundu, Ben Stoffelen, Kaili Wang, Peter Vrancx, Ludovic Denoyer
The paper introduces DNBNet, a Debiased Neural Basis-Function Network designed for irregular time series forecasting. It addresses two main limitations of existing methods: asymptotic bias from ignoring timestamp sampling density and limited adaptability of predefined basis functions. DNBNet employs importance sampling to correct bias, neural‑network parameterized basis functions for flexibility, a multi‑scale decomposition with mass‑aware fusion for sparse data, and a dual‑branch decoder, achieving strong performance across diverse real‑world datasets.
By Rongwen Li, Changjian Chen
The paper introduces Role-Conditioned Sub-Token Routing (RoleSub), a method that compresses the value representations of retained tokens in Vision‑Language‑Action models. By partitioning each token’s value into orthogonal groups and routing them based on token features, latent roles, and language context, RoleSub can compress both visual and language representations without discarding tokens. Experiments on OpenVLA‑OFT‑7B show that, at matched visual‑KV budgets, RoleSub outperforms token‑only control in most settings and can reduce total KV to 9.2–11.3% of the original while maintaining strong control performance.
The paper investigates whether reader-specific differences in retrieval‑augmented generation (RAG) reflect reusable structure or merely input‑local interactions. By fixing query, evidence, task, scoring, and intervention, the authors find that nine readers disagree on the effect sign in 33% of cases, with reader×query interactions explaining 29.8% of utility variance. They further decompose heterogeneity into evidence activity, ordinal preference, and conditional signed direction, discovering that ordinal reader geometry is stable across multiple settings while signed geometry is task‑bounded, yet stable ordinal similarity does not predict cross‑reader intervention transfer.
MoE-ViE introduces a Mixture-of-Experts vision encoder that scales efficiently for image and video understanding, outperforming dense counterparts across various sizes. The study shows fine‑grained MoE topologies provide significant gains, and proposes an auxiliary‑loss‑free balancing variant and a specialized MoE kernel to reduce inference latency. With frame‑level distillation and a novel freezing mechanism, the largest MoE‑ViE model matches state‑of‑the‑art zero‑shot performance while being 1.7× larger and 76% faster, and it outperforms other encoders when paired with a language model on both image and video benchmarks.
arXiv:2608. 15762v1 Announce Type: cross Abstract: Container-granularity scheduling leaves abundant short-lived idle slices within containers unexploited.
By Weinan Liu, Zeyuan Ding, Dian Ding, Chengcheng Wan, Lu Tang, Guangtao Xue, Jiwu Shu, Yiming Zhang
arXiv:2608. 15632v1 Announce Type: cross Abstract: Neural representations have become a central tool for studying the internal mechanisms of modern AI models, yet their complex high-dimensional structure makes them difficult to interpret.
By Yehonatan Avidan, Daniel D. Lee, Haim Sompolinsky
arXiv:2608. 15787v1 Announce Type: cross Abstract: Two Mixture-of-Experts (MoE) forward passes can share every weight yet route the same token through different experts.
By Cedric Caruzzo, Donggeun Yoo, Tae Soo Kim