The paper introduces DEX-Comp, a two‑stage training method for soft context compression in Retrieval‑Augmented Generation (RAG). First, a pure distillation warm‑start trains the compression model on correct responses from an uncompressed RAG. Then, hard exploration uses reinforcement learning on queries where the uncompressed RAG fails, encouraging better computation patterns for compressed representations. Experiments on five open‑domain QA benchmarks show that DEX‑Comp compresses retrieved contexts 16×, speeds inference 4×–24×, and matches or surpasses the uncompressed RAG baseline across various retrieval depths.
By Shuyu Guo, Shuo Zhang, Zhaochun Ren
The study explores non‑smooth activation functions—chaotic, stochastic, and fractal—in echo state networks, testing 36,610 reservoir configurations. It finds that functions like the Cantor function preserve the Echo State Property (ESP) even at spectral radii up to 10, outperforming traditional smooth activations in convergence speed. The authors introduce a theoretical framework for quantized activations, defining a Degenerate Echo State Property (d‑ESP) that implies the traditional ESP, and show that preprocessing topology, rather than continuity, governs stability.
By Rae Chipera, Jenny Du, Irene Tsapara
The paper presents a deep learning approach that converts nanopore resistive pulse signals into scaleograms using continuous wavelet transforms, enabling the classification of peptides as an image‑classification problem. On a dataset of 42 peptides, the method achieves an 82% macro‑averaged accuracy, outperforming previous descriptor‑based techniques by 8.6 percentage points. The models also remain accurate after significant weight pruning and 8‑bit quantization, making them suitable for deployment on embedded sensing hardware.
By Julian Ho{\ss}bach, Samuel Tovey, Sandro Kuppel, Tobias Ensslen, Jan C. Behrends, Christian Holm
VLA-Precision introduces an efficient real‑world online reinforcement learning framework for vision‑language‑action (VLA) models, featuring the Asymmetric Co‑Bootstrapping (ACoB) algorithm and the ACoB‑Stream architecture. ACoB uses asymmetric co‑bootstrapping across timescales to rapidly improve policy performance while refining value estimates, thereby reducing policy drift. ACoB‑Stream enables large VLA models to run with up to 10.9× higher throughput and computational efficiency, achieving a 98.3 % mean success rate on nine high‑precision chemistry tasks in under 46 minutes per task.
By Chenyu Su, Zhaolong Shen, Yuan Qian, Chen Qian, Rui Zhang, Feng Yan, Weixing Chen, Fei Zhang, Jiamin Wang, Shuang Cong, Weiwei Shang
The paper introduces MIRC, an overfitted image codec that quantizes and entropy‑codes all components—including latents, synthesis network, and entropy models—within a single end‑to‑end rate‑distortion framework inspired by NVRC. It adds a multi‑scale representation with cross‑stage parameter sharing to capture cross‑scale redundancy, yielding a 10.5 % BD‑rate saving over VVC on the CLIC2020 professional set. MIRC offers multiple configurations ranging from 1.2 to 2.9 kMAC per pixel, allowing decoding complexity to be tuned to deployment needs.
By Tianhao Peng, Ho Man Kwan, Fan Zhang, Shan Liu, David Bull
Scale-QLoRA introduces a method for merging LoRA adapters into native 4‑bit microscaling language models without altering the quantized code plane. By training only the per‑block scale field and freezing the E2M1 code, the merged model remains code‑invariant and can be deployed as a single checkpoint. Experiments on four models and tasks show that Scale‑QLoRA achieves accuracy‑lossless merging comparable to merge‑aware QAT‑LoRA, while offering benefits such as exact rollback, code‑plane deduplication, and a 125× faster scale‑only task swap.
By Tung-Ling Li, Jiale Huang, Lee-Chi Wang, Janaki Ram Gotei
The paper documents a growing trend of removing safety guardrails from open-weight AI models, profiling the ecosystem that produces, redistributes, and applies these models. Between January 2024 and March 2026, 3,471 original uncensored models were identified on HuggingFace, each re‑packaged an average of 2.4 times, with three actors responsible for 52 % of all 8,164 compressed redistributions. After quantization and mirroring across platforms such as Ollama, these models persist even when upstream versions are removed, and 25 % of 1,643 GitHub applications that integrate uncensored large language models were classified as explicitly malicious.
By 10a Labs, :, Juliette Garcia, Hailey May, Bobby McKenzie, David Pham, Matthew Swain, Joshua Valdez, Corie Wieland, Zachary Yahn
arXiv:2609. 04575v1 Announce Type: cross Abstract: Modern fine-grained Mixture-of-Experts (MoE) models route each token to a small number of experts and renormalize their router probabilities.
By Xing Chen, Hengshuai Yao
The paper introduces a communication‑efficient personalized federated learning framework that uses layer‑wise multi‑threshold random sketching. By assigning each neural network layer its own set of quantization thresholds, the method adapts to layer‑specific parameter distributions and provides a finer low‑bit representation than single‑threshold one‑bit compression. This approach supports bidirectional communication with compact sketches and improves the communication‑accuracy tradeoff over existing one‑bit methods.
By Xu Zhang, Xingyu Hou, Jiacheng Cheng, Kaiyuan Feng, Maoguo Gong
The paper introduces a framework for instruction‑guided 3D editing that does not require paired 3D supervision. It distills visual, semantic, and geometric knowledge from foundation models into a 3D editing model using a differentiable rendering pipeline, guided by a 2D visual prior from an image editing model and a semantic prior from a Vision‑Language Model. A 3D‑aware Distribution Matching regularization is added to prevent geometric collapse and ensure realistic 3D outputs, leading to superior instruction fidelity and cross‑view consistency compared to state‑of‑the‑art baselines.
By Hao Wen, Weibin Yun, Hongxing Fan, Haotian Lu, Rui Chen, Zehuan Huang, Lu Sheng
Weather-Conditioned Depth Anything (DA‑W) is a new framework that enhances monocular depth estimation models, like the Depth Anything series, to perform robustly under adverse weather conditions such as fog, rain, snow, and low‑light. It achieves this by disentangling style from content: a Style Filter extracts weather‑specific embeddings from a curated mix of real and synthetic degradation data, which are then injected into the backbone via a lightweight, zero‑initialized adapter. The adapter is trained with pseudo‑label distillation and alignment, enabling a single unified model to adapt to diverse weather scenarios while preserving its generalization on clean data, and it achieves state‑of‑the‑art performance with an average 3.7% improvement in AbsRel on weather benchmarks.
By Zhaoming Xu, Chan-Wei Hu, Kuan-Ru Huang, Zihao Zhu, Renjie Li, Yang Zhou, Zhengzhong Tu
The paper proposes a new strategy for merging layers in deep neural networks, enabling depth compression without requiring an analytical solution for convolutions with padding and without increasing kernel size. This approach addresses limitations of previous methods that struggled with padded convolutions and larger kernels, and it is validated across various architectures and datasets with measured inference speed-ups on embedded platforms.
By Petro Shulzhenko, Gabriele Spadaro, Enzo Tartaglione
The paper demonstrates that layer dropout, also known as stochastic depth, can be effectively used in state‑of‑the‑art large language model (LLM) training. By optimizing the layer distribution, schedule, and optimizer settings, the authors show that layer dropout can reduce training loss while saving up to 25 % of training FLOPs. Additionally, layer dropout enables post‑training optimizations such as early exit and self‑speculative decoding, achieving up to 1.5× inference speedup with negligible accuracy loss across models ranging from 271 M to 8.2 B parameters and datasets up to 160 B tokens.
By Mostafa Elhoushi, Alex Pretko, Nolan Dey, Bin Claire Zhang, Gavia Gray, Gurpreet Gosal, Abdulrahman Mahmoud, Shane Bergsma, Joel Hestness
The paper investigates how to recover language model quality lost when using low‑bit key–value caches for autoregressive decoding. By keeping the quantizer fixed and distilling the full‑precision cache behavior into low‑rank Q/K/V projection updates, the authors demonstrate that 4‑bit affine‑cache adapters recover roughly 54 % of the perplexity gap on TinyLlama‑1.1B and 76 % on Gemma‑4‑12B, while preserving most long‑context retrieval. Even a 2‑bit rank–token sweep can dramatically reduce TinyLlama’s perplexity, though it only partially restores retrieval performance.
By Seifeldin Abdellatif
The paper introduces a compact BART-based sequence‑to‑sequence model for low‑latency spell correction of Japanese music search queries, addressing challenges posed by four writing scripts. It employs a script‑aware synthetic misspelling pipeline that blends keyboard‑layout models, phonetic confusion priors, consonant alternations, and kana case errors, and normalizes mixed‑script titles to a single canonical script to reduce hallucinations. Experiments show the model achieves 41.09% exact‑match accuracy and 11.62% CER, outperforming edit‑distance baselines while keeping inference latency under 4 ms on a single GPU.
By Anshul Garg, Pavni Tandon, Karan Bhukar, Tanmay Khandelwal, Ujjal Kumar Dutta
The paper investigates regression with bounded responses, comparing two learning frameworks: model selection aggregation, which requires improper algorithms to achieve minimax excess risk, and universal learning, where empirical risk minimization suffices for exponential learning rates. For finite hypothesis classes, the authors show that the $Q$-aggregation estimator simultaneously attains minimax optimal tails and exponential universal rates, while other common estimators fail to do so. For countably infinite classes, they prove an inherent trade‑off between exponential universal and minimax uniform rates, resolved by combining optimal algorithms from each framework via $Q$-aggregation.
By Mikael M{\o}ller H{\o}gsgaard, Patrick Rebeschini, Tobias Wegel
The paper presents a unified compression framework for Vision Transformers aimed at on‑device plant disease detection in resource‑constrained agricultural settings. It combines Hessian‑Balanced Adaptive Block Pruning, quantization, and attention‑based knowledge distillation, evaluating each component separately before integrating the best performers into a deployment pipeline. On a chilli disease dataset, the compressed models achieve accuracy comparable to the FP32 baseline while reducing model size by 74‑98 %, and the full pipeline attains a 54.5× size reduction to 6.01 MB with 95.13 % accuracy.
By Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith, Gangireddy Rahul Jogi, Sudheesh Manalil, Arnab Raha, Amitava Mukherjee, Parthasarathy Seethapathy, G. Gopakumar
The paper presents a two-level framework for scalable trade‑up recommendation. Level 1 distills large‑language‑model reasoning into a compact, non‑generative student that classifies product pairs using only precomputed embeddings, achieving high AUC on a benchmark. Level 2 applies product‑type test‑time training to fine‑tune lightweight adapters, further improving performance while keeping inference fast and inexpensive.
By Siliang Liu, Mohammad Ghasemi, Sapan Patel, Amin Banitalebi-Dehkordi
The paper introduces juniper, a discrete denoising diffusion model that performs compositional backmapping from coarse‑grained (CG) beads to detailed molecular graphs. By conditioning on the octanol–water partition free energy, the model generates valid and unique molecules that closely match the target free‑energy distribution for two‑bead systems. This approach enables the conversion of CG screening results into candidate atomistic molecules for further study or synthesis.
By Luis Itza Vazquez-Salazar, Tristan Bereau
The paper investigates how prefix caching, a default optimization in open‑source LLM serving stacks, affects reproducibility when combined with weight quantization. Experiments on an eighty‑episode multi‑turn agentic tool‑use workload show that enabling the cache causes the agent’s trajectory to change in 36.2 % of episodes at 16‑bit precision and 75.0 % at 4‑bit precision, while disabling the cache yields perfectly reproducible runs. The study identifies specific cache‑related settings that drive run‑to‑run divergence and demonstrates that cached serving is deterministic only when the cache state is preserved, which is not the case in typical deployments.
By Aditi Patodiya