MOMAT (Mixture of Multiple Atlases) is a hardware‑enhanced safety framework designed to defend quantized large language models (qLLMs) on edge devices against jailbreak attacks. It uses a collection of semantic atlases—each containing harmful or benign sample clusters and policy templates—to perform domain‑localized Retrieval‑Augmented Generation. A lightweight Mixture of Experts detector evaluates top‑k similarity features retrieved by a Compute‑in‑Memory (CiM) accelerated engine, achieving a 4.69 × 10⁶‑fold speedup and a 2.5 × 10⁵‑fold energy reduction compared to DRAM‑based baselines while matching state‑of‑the‑art defense performance.
By Boyang Li, Bingyu Shen, Weihao Hong, Zhiyuan Jiang, Xinlei Guan, Yan Ma, Miles Q. Li, Yi Sheng, Ruiyang Qin
The paper introduces TALON, a Task‑Adaptive LoRA‑Teacher framework for Few‑Shot Class‑Incremental Learning. TALON assigns a dedicated LoRA‑Teacher to each incremental task, then distills the frozen teachers into a single LoRA‑Student via Ensemble Knowledge Transfer, using a semantic‑guided weighting scheme to reduce forgetting and overfitting. Experiments on four FSCIL benchmarks show that TALON matches or surpasses state‑of‑the‑art accuracy while using up to 33× fewer deployment parameters and cutting inference time by 41.7%.
By Hongwei Zhao (School of Computer Science,Engineering, Beihang University), Rui Liu (School of Computer Science,Engineering, Beihang University), Yansong Liu (School of Computer Science,Engineering, Beihang University), Zhiyuan Zou (School of Computer Science,Engineering, Beihang University), Yong Chen (School of Computer Science, Beijing University of Posts,Telecommunications)
The paper introduces RATIO, a framework for improving quantized reasoning models by identifying overthinking tokens and applying token-specific penalties. It uses Quantization-aware Reasoning Behavior Analysis to detect problematic tokens and Token-Specific Penalty Determination to assign penalties without extra training. Experiments show RATIO outperforms existing methods, boosting accuracy by up to 9.8 points and shortening chain-of-thought length by up to 51.3%.
By Chengzhu Bao, Xianglong Yan, Tianao Zhang, Jiaqi Chen, Shaoqiu Zhang, Yulun Zhang
The paper introduces a fault‑tolerant budget conservation framework for distributed multi‑agent delegation, where budgets are represented as exclusive escrow credits that traverse a delegation DAG. It details how each branch converts credit into a reservation tied to lineage, epoch, and idempotency, persists a signed dispatch permit, and ensures that uncertain effects remain charged until settlement or retirement. The authors prove properties such as ownership partition, ledger conservation, and at‑most‑once settlement, and validate the mechanism through TLA+ checks, a JavaScript explorer, and crash‑injected SQLite experiments.
By Genliang Zhu, Chu Wang
ProtoFlow is a new multivariate time series forecasting framework that combines vector‑quantized autoencoding with prototype‑guided flow matching. It maps sequences into a discrete latent space, constructs a structured prior from the learned VQ codebook, and trains a DiT‑based rectified flow to transport samples from this prior to future latent representations conditioned on past observations. By replacing generic Gaussian noise with a learned prototype prior, ProtoFlow eliminates autoregressive rollout mismatch and achieves faster training convergence while delivering superior forecasting performance on benchmark datasets.
By Shibo Feng, Wanjin Feng, Yang Qiu, Deheng Ye, Peilin Zhao, Chunyan Miao
The paper presents a billing‑aware neural text‑to‑speech system for serverless CPUs, focusing on minimizing CPU‑seconds and GB‑seconds rather than just throughput or latency. By using request‑sized concurrent inference and a reclaimable instance lifecycle, the system limits per‑request CPU parallelism and releases idle memory while keeping the server process alive. On the Kokoro‑82M benchmark, it achieves 2.71 audio‑seconds per CPU‑second versus 0.89 with ONNX Runtime, cuts cost per audio‑hour from $0.0631 to $0.0153, and reduces idle billed memory from 8.7 GB to 1.33 GB, with faster restoration times.
By Pakorn Nathong, Kunat Pipatanakul
The paper demonstrates that using a single activation function across all nodes in artificial neural networks imposes hard limits on evolutionary search, particularly for sparse evolved substrates. By evolving per-node activation functions from an 18-function palette, the authors show that oscillatory functions can solve parity problems at all tested scales, while monotonic functions fail beyond the simplest case. The study reveals that the choice of activation functions, beyond topology and weights, critically influences what evolutionary search can achieve, and that heterogeneous assignments discovered via indirect encoding are unlikely to be selected manually.
By Romain Claret, Michael O'Neill, Paul Cotofrei, Kilian Stoffel
The paper investigates four ways to grow a classifier—adding a tree level, a hidden unit, a leaf split, and a statistically significant split—under a fixed protocol for tree‑structured and constructive models. It shows that the most natural method of deepening a soft decision tree by duplicating a leaf’s class distribution leaves the gradient of new gates identically zero, preventing learning, and proposes a small random perturbation as a fix. The other three growth decisions each provide a distinct benefit: fitting a new hidden unit to residual error yields a smaller network, splitting the leaf with the largest expected error adds sparsity, and requiring statistical significance before splitting adds no value and reduces accuracy.
By Cagri Temel
DriftOPD is a teacher‑free, rollout‑free framework that performs sequence‑level on‑policy distillation of continuous Vision‑Language‑Action (VLA) action experts. It decomposes the sequence‑level reverse‑KL divergence into a chunk‑level reverse‑KL term and a future‑potential term, optimizing them with a one‑step drifting objective and a Q‑function critic learned from offline demonstrations. Experiments on multiple VLA architectures in simulation and real‑world manipulation show that DriftOPD outperforms existing one‑step distillation baselines while matching the task success of multi‑step teacher policies.
By Youngjun Jun, Kyumin Choi, Youngmin Kim, Seonghyun Jin, Sunwoo Park, Jangho Park, Jong Chul Ye
MEGA is a new framework that extracts object-level, watertight meshes from 3D Gaussian Splatting (3DGS) scenes. It uses a segment-then-mesh approach, leveraging Spatial Visual Distillation (SVD) to sample diverse camera views of each segmented object and train a mesh reconstruction model with photometric supervision. Experiments on popular benchmarks show that MEGA outperforms existing methods in accurately recovering object-level 3D occupancy and supports complex physical interactions by combining high-quality meshes with photorealistic 3DGS rendering.
By Liwei Liao, Yingkui Zhang, Qianqian Tong, Ronggang Wang
Omni-Embed-Mini is a 0.9B‑parameter model that embeds text, speech, audio, images, video, and visually‑rich documents into a single shared cosine space without updating any text‑side parameters. It uses a dense cascaded caption as a teacher signal, allowing the teacher and student to share identical backbone weights and requiring only lightweight projectors and phased LoRA adapters for alignment. The model achieves strong text retrieval performance (49.57 nDCG@10 on MTEB‑v2 BEIR‑8) while extending to five additional modalities and is significantly smaller than other open omni‑modal embedders.
By Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly, Ivan Laptev, Hisham Cholakkal
The paper proposes a new architecture for large language model (LLM) agents that enhances spatial understanding by combining geometrical tools with an LLM orchestrator in grid‑world environments. It first gathers geodesic trajectories, vector‑quantizes them to create a representative subset, and then has the LLM label each trajectory with a natural language description, turning them into reusable tools. During operation, the LLM selects the appropriate tool based on the current state and goal, while low‑level control executes the chosen trajectory, enabling efficient decision‑making in a partially observable 2D grid setting.
By Gabriel Turinici
Fold'EM is an inference-time framework that directly combines protein generative model priors with cryo‑EM particle images to recover atomic structures, bypassing the traditional density reconstruction step. It can determine structures from a small number of particles, jointly infer orientations in an ab‑initio setting, and resolve distinct conformational states in heterogeneous samples without separate map reconstruction. The method demonstrates accurate atomic models on both synthetic and experimental datasets, showing promise for low‑sample and low‑population conformational analysis.
By Advaith Maddipatla, M\"art-Erik M\"aeots, Marco Pegoraro, Nikolaus Dr\"ager, Roberto Covino, Sanketh Vedula, Martin Pacesa, Alex M. Bronstein
arXiv:2606.22826v2 Announce Type: replace
Abstract: Evaluating LLMs across many model variants---quantized, fine-tuned, or deployment-specific---requires running large benchmarks repeatedly, a proces...
By Devleena Das, Rajeev Patwari, Vikram Kumar Bukka, Nithin Kumar Guggilla, Elliott Delaye, Ashish Sirasao
arXiv:2609.39068v1 Announce Type: new
Abstract: Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. Although sparse attention reduces these...
By Jitai Hao, Quansheng Gu, Qiang Huang, Jun Yu
arXiv:2609.39131v1 Announce Type: new
Abstract: Large language model (LLM) serving requires substantial memory to store model weights and KV caches. As models grow larger and contexts become longer,...
By Zack Yu, Chloe Wong, Coleman Hooper, Minjae Lee, Wonjun Kang, Youngjin Cho, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami
arXiv:2609.39223v2 Announce Type: new
Abstract: Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive p...
By Weili Xu, Jisen Li, Yuqing Jian, Chenxi Li, Zhizhou Sha, Yifan Yu, Qingyang Wu, Chenfeng Xu, Zhongzhu Zhou, Tianyi Zhang, Ben Athiwaratkun
arXiv:2609.39275v1 Announce Type: new
Abstract: On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher'...
By Zixiang Ni, Zhuo Hu, Renjie Cao, Weijie Ren, Binqin Shi, Weijia Zhang, Shuheng Cao, Zhicheng Shi, Zhenhao Zhang, Haomin Wen, Zhiyuan Hu
arXiv:2609.39329v1 Announce Type: new
Abstract: Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compressi...
By Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi, Donggyun Kim, Seunghoon Hong
arXiv:2609.39338v1 Announce Type: new
Abstract: Knowledge distillation transfers knowledge by encouraging a student to match a teacher's predicted class probabilities. These probabilities express not...
By Qianfeng Yuan, Wenbing Tao