Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv:2609.05641v1 Announce Type: cross Abstract: We consider the problem of minimizing error in quantized matrix multiplication $C=AB$. Scalar quantization of the factors introduces rounding errors...
arXiv:2601.03163v2 Announce Type: replace Abstract: Background and Objective: Precise and scalable instance segmentation of cell nuclei is a fundamental prerequisite for computational pathology, yet...
arXiv:2609.10311v1 Announce Type: cross Abstract: The lottery ticket hypothesis posits the existence of winning tickets: sparse subnetworks that, when trained in isolation from their original initial...
The paper introduces RouteOPD, a new on‑policy distillation method that treats teacher‑student disagreement as a probability transport problem. By decomposing disagreement into excess sources and deficit destinations, RouteOPD pairs them explicitly and optimizes pairwise log‑odds toward targets derived from a bounded teacher potential. Experiments on four teacher‑student configurations and four mathematical‑reasoning benchmarks show that RouteOPD consistently outperforms sampled reverse‑KL OPD, achieving higher routing fidelity and lower background leakage.
arXiv:2603.04949v2 Announce Type: replace Abstract: As web agents close the gap with humans on benchmarks, one question arises: Do today's agents perform just as well on tomorrow's web? We introduce...
arXiv:2609.06195v1 Announce Type: new Abstract: This paper proposes a transferable Map of Dynamics (MoD) framework that generalizes to unknown environments using only egocentric 3D LiDAR point clouds...
The paper introduces Stable-MM-R1, a framework that stabilizes reinforcement learning for multimodal reasoning by addressing training instability and entropy collapse. It proposes Potential‑Aware Query Mining (PAQM) to filter data toward high‑potential samples and Hybrid Stratified Replay (HSR) to restructure batches using path entropy and reward stratification, reusing stability anchors and hard negatives. The method demonstrates superior performance on complex reasoning tasks compared to strong baselines.
arXiv:2609.09300v1 Announce Type: new Abstract: Video understanding demands a convergence of complementary capabilities across perception, temporal understanding, and complex reasoning, which are dif...
The paper critiques current evaluations of efficiency methods for large language model–based multi‑agent systems, arguing that reported gains are often inflated by method‑specific prompts and starting topologies. It introduces a controlled, MAS‑demanding diagnostic benchmark that standardizes the backbone model, agent registry, and runtime, and systematically varies topology, scale, depth, and tool use. The authors find that many claimed efficiency improvements are setup‑dependent, sometimes stemming from structural collapse or random pruning rather than genuine, robust gains.
CALM introduces a smooth trust gating mechanism for decentralized federated learning, replacing hard filtering of teacher models with class‑wise, sample‑wise, and label‑based weighting. It allows clients with heterogeneous architectures to distill knowledge from peers without a central server or shared data, even under severe non‑IID label skew. Experiments on CIFAR‑10, SVHN, OrganAMNIST, and Google Speech Commands show that CALM consistently outperforms uniform and hard‑filtered distillation and matches or exceeds other heterogeneous‑FL methods.
arXiv:2609.09863v1 Announce Type: new Abstract: Choosing a deep learning architecture for label-free single-cell classification remains an open question, with microscopy benchmarks reporting conflict...
The paper introduces GaugeLasso, a method that applies symmetric group‑lasso penalties to transformer channels during training, enabling entire tensor slices to be zeroed out while maintaining dense tensors for GPU efficiency. By calibrating channel penalties based on inference utility per compute, the network self‑organizes into depth‑dependent structural profiles that can be dramatically smaller than the original architecture, achieving up to 255‑fold compression on a polynomial division task and outperforming hand‑designed baselines on language modeling and autoencoding benchmarks. The approach also accelerates training and reveals over‑provisioned axes that guide subsequent design iterations.
arXiv:2609.08609v2 Announce Type: replace Abstract: Tracking semantic change in low-resource languages across extensive historical timelines presents significant challenges due to data scarcity and t...
The paper introduces TokCode, a token encoding framework that enhances robustness in generative semantic communication by restructuring redundancy in the semantic domain. TokCode leverages a lightweight adapter to transform a large language model into a token encoder, avoiding the need for a dedicated deep model. A channel-quality-aware distillation method (CADET) trains the adapter across diverse erasure rates, producing a reconfigurable low‑rank adapter that enables efficient reinforcement learning and achieves significant improvements in image similarity over existing receiver‑side recovery benchmarks.
Osprey is a target‑agnostic pre‑training method that bootstraps draft models for speculative decoding from existing small language models. By pruning to a shallow backbone, restoring language‑modeling capability with next‑token pretraining, and adapting via vocabulary alignment and distillation, Osprey reduces per‑target work to a lightweight adaptation step. Experiments show that a single Osprey backbone improves mean acceptance length by up to 22.7% and increases tokens per second by 17.5% across several large target models, especially on out‑of‑domain and multilingual data.
The paper introduces a data‑centric pipeline for post‑training language models on financial reasoning tasks. It mines open‑source reasoning traces, distills financial instruction data, and generates knowledge‑graph‑guided question‑answer pairs, then filters examples with lightweight classifiers and applies reinforcement learning with rule‑based verifiers. Experiments on FINESSE‑Bench show that retention‑aware adaptation—self‑distilled fine‑tuning and model merging—outperforms ordinary supervised fine‑tuning, improving accuracy by up to 3.0 points and avoiding regressions.
arXiv:2609.08638v1 Announce Type: cross Abstract: An action chunk can span several stages of a manipulation task, yet a label for its first step describes only the current stage. We introduce Chunk-A...
arXiv:2609.08307v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires bala...
SequenceO1 is an end‑to‑end framework that enables ultra‑long (up to 100K interactions) sequence modeling for recommendation systems. It compresses raw user histories into a fixed‑size sketch using Sketch Attention and then models short‑term and long‑term interests with Target‑to‑History Cross Attention. The system incorporates low‑rank caching, batching, pipeline lift, and a FlashSA kernel to keep training and inference efficient, achieving consistent offline and online performance gains when deployed at full traffic on Douyin.
arXiv:2609.08123v1 Announce Type: cross Abstract: A robot that can be taught a new task from a handful of demonstrations has to work out for itself what it still cannot do, and then ask for exactly t...