Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv:2602.09691v2 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is a technique to compress a larger teacher system into a smaller student. In machine translation, KD is commonly...
arXiv:2609.00407v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models enable efficient scaling of large language model (LLM) inference but suffer from substantial data-movement overhead w...
arXiv:2608.12018v2 Announce Type: replace Abstract: Neural Machine Translation (NMT) and Large Language Models (LLMs) excel at cross-lingual tasks but often fail to capture intra-lingual morphologica...
Residual sparsification via output importance (PARSER) is a new compression technique for mixture-of-experts large language models that shifts the compression objective from minimizing isolated matrix errors to preserving the expert output error. By introducing output importance, PARSER measures each residual’s contribution to the final expert output and compresses accordingly. Experiments show that PARSER reduces the accuracy gap to the uncompressed model by 1.41× on Qwen and 1.44× on DeepSeek while achieving the same peak memory reduction.
ARISE‑RL is a full‑cycle self‑evolution framework that couples a task/rubric generator with a reasoning solver, enabling open‑ended agents to learn via reinforcement learning without relying on gold answers. The generator creates tool‑grounded rubric criteria and rewards itself for producing valid, intermediate‑difficulty tasks, while the solver improves through fine‑grained rubric satisfaction signals and multi‑step reasoning. The framework also introduces Reward‑Gated Self‑Evolution Distillation to selectively distill memory‑augmented policies, reducing distribution mismatch, and it is evaluated on the new ECR‑Bench rubric suite, achieving state‑of‑the‑art performance across all benchmarks.
The paper introduces the Hierarchical Semantic Distillation Network (HSDN), a method for compressing chain-of-thought reasoning in financial contexts while maintaining accuracy and logical coherence. HSDN uses semantic segmentation, dependency graph construction, dual encoder importance scoring, constrained segment selection, and local boundary rewriting to reduce the length of intermediate reasoning traces. Evaluated on the AFAC2025 benchmark, HSDN achieves 91.0% accuracy with a 68.4% compression rate, outperforming strong baselines in overall score and reasoning coherence.
arXiv:2603.10359v2 Announce Type: replace Abstract: Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitations of rejection s...
SinkPruner is a training‑free framework that prunes visual tokens for multimodal large language models by first removing high‑norm redundant tokens with a visual sanitizer and then selectively keeping tokens that align with the text query using a text‑guided pruner. The coarse‑to‑fine design reduces attention sink and dispersion, enabling an 89% token reduction while preserving 96.5% of LLaVA‑1.5’s performance and 91.8% of Qwen2.5‑VL’s performance across twelve image‑language and four video‑language benchmarks. The visual sanitizer also improves existing pruning methods, showing strong transferability.
The paper studies how to compress Chain-of-Thought (CoT) reasoning traces for smaller models. It examines three compression dimensions—importance criterion, restructuring level, and compression budget—across Math and General domains and Long/Short CoT regimes. Findings show that step-level pruning works best for shared reasoning backbones, token-level pruning needs symbol-aware signals, domain-specific restructuring effects differ, and training-time compression may not reduce inference cost, especially for Long-CoT students.
arXiv:2609.00555v1 Announce Type: cross Abstract: Language-driven dexterous grasp models, such as DextER, perform well when instructions specify where to grasp, but we find they fail systematically w...
arXiv:2609.00901v1 Announce Type: new Abstract: Modifying the illumination of driving images is a fundamental challenge, as most datasets are captured at specific times of day. Existing methods rely...
arXiv:2609.00706v1 Announce Type: new Abstract: The SAIR Mathematics Distillation Challenge on Equational Theories asks a solver to classify whether one magma identity implies another and, for either...
arXiv:2609.00066v1 Announce Type: cross Abstract: NVFP4 is an efficient microscaling format for low-bit inference, but activation outliers can still degrade quantization accuracy within NVFP4 blocks....
arXiv:2609.00198v1 Announce Type: new Abstract: Modern explicit-time generative models, such as Flow Matching [Lipman et al., 2023] and Rectified Flow [Liu et al., 2023], are typically derived top-do...
arXiv:2609.00776v1 Announce Type: cross Abstract: Digital interfaces are traditionally implemented through intermediate representations such as code, requiring their appearance and behavior to be spe...
arXiv:2609.00224v1 Announce Type: cross Abstract: Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existi...
arXiv:2609.01224v1 Announce Type: new Abstract: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens. Existing m...
The paper investigates why supervised fine‑tuning (SFT) of large language models leads to increased hallucinations of factual information. It proposes a self‑distillation SFT approach that regularizes output‑distribution drift to preserve pre‑training knowledge, and shows that freezing parameter groups can reduce hallucinations when new knowledge is unnecessary. Experiments attribute the main cause to interference among overlapping semantic representations, which self‑distillation mitigates, and an associative‑memory model explains the forgetting dynamics.
arXiv:2609.00059v1 Announce Type: cross Abstract: Materials property prediction remains difficult in low-data settings, where many target properties are supported by only a limited number of labeled...
The paper investigates KV cache offloading for long‑context large language models, focusing on tasks that require extensive information extraction from the prompt. The authors introduce the Text2JSON benchmark, a highly context‑intensive task that demands structured knowledge extraction from raw text, and evaluate modern KV offloading techniques on this benchmark and other similar tasks. Their experiments on Llama 3 and Qwen 3 reveal significant accuracy degradation, attributing it to low‑rank key projection and unreliable landmarks, and propose a simpler strategy that markedly improves performance across multiple LLM families.