Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv:2604.11501v2 Announce Type: replace-cross Abstract: Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression shoul...
arXiv:2609.06712v2 Announce Type: replace Abstract: Diffusion Transformers (DiTs) achieve strong video generation quality, but their dense spatiotemporal self-attention scales quadratically with sequ...
arXiv:2609.10346v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existin...
arXiv:2609.08368v1 Announce Type: new Abstract: We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each st...
arXiv:2609.06557v1 Announce Type: new Abstract: Large language models (LLMs) are often considered fragile under aggressive sparsification, and maintaining reliable performance typically requires stic...
The paper introduces a structured post‑training pruning method for vision and language transformers called Damage‑Aware Bandit Pruning. It treats the selection of functional units (attention heads and MLP channel groups) as a multi‑armed bandit problem, using paired damage (masked loss minus base loss) as a reward to guide either UCB or Thompson Sampling policies. Experiments on a range of models (GPT‑2, OPT, Pythia, Qwen2.5, SmolLM2, ViT‑B/16, DeiT‑Tiny, Swin‑Tiny) show that the bandit approaches generally reduce degradation compared to budgeted‑greedy baselines, with statistically significant improvements in most comparisons.
The paper proposes a new algorithm for generating Semantic IDs that are both unique and preserve the structure of the original embedding space. By employing bottom‑up clustering, the method maintains local structure, leading to higher clustering quality. This improved structure enhances the utility of the Semantic IDs for downstream generative retrieval tasks.
ASDA (Automated Skill Distillation and Adaptation) is a framework that improves large language models on financial reasoning tasks without fine‑tuning. It works by having a teacher model analyze a student’s failures, cluster errors, and generate structured skill artifacts—reasoning procedures, code templates, and worked examples—that are injected during inference. On the FAMMA benchmark, ASDA boosts arithmetic reasoning by up to 17.33% and non‑arithmetic reasoning by 5.95%, outperforming existing training‑free methods.
The paper presents a second‑order theory of quantization noise for matrix multiplication, characterizing quantization formats by the variance they assign to each element. It derives a closed‑form signal‑to‑noise‑ratio law for floating‑point rounding, introduces an upper bound κ* that cannot be exceeded by any function‑preserving linear transform, and proposes KBBQ—a method that parameterizes how closely a transform approaches this bound. Experiments on W4A4 across four base models and two FP4 formats show that KBBQ outperforms the previous state of the art without extra deployment‑time computation.
arXiv:2609.06430v1 Announce Type: new Abstract: We study the identity straight-through estimator (STE) for training a two-layer binary-activation network with hinge loss from the perspective of Stati...
arXiv:2609.06483v1 Announce Type: new Abstract: This study focuses on the scientific problem of understanding internal factors that govern the diverse performance degradation of deep neural networks...
The paper evaluates entropy-based pruning for compressing Chain-of-Thought (CoT) reasoning in large models. Across multiple models and tasks, low- and high-entropy step selection shows no advantage over random pruning, and low-entropy token retention only helps on mathematical benchmarks due to the low entropy of numeric tokens. Patching a few CoT tokens with their original activations restores near-perfect performance, indicating that task information is distributed throughout the entire reasoning chain rather than concentrated in a small set of tokens.
arXiv:2609.08318v1 Announce Type: cross Abstract: The transition from human-centric assistance to Autonomous Software Engineering (ASE) agents has enabled the resolution of complex real-world SE task...
arXiv:2609.06154v1 Announce Type: new Abstract: One-shot federated graph learning generally aims to train Graph Neural Networks (GNNs) across clients with disconnected subgraphs in a single communica...
The paper evaluates the scalability and adversarial generalization of Natural Language Inference (NLI) models trained with Group Relative Policy Optimization (GRPO) for Chain-of-Thought learning. By fine‑tuning 7B, 14B, and 32B language models with LoRA and QLoRA, the authors show strong performance on standard and adversarial NLI benchmarks, with the 32B model outperforming supervised baselines on adversarial sets. Using AWQ quantization, the 32B model fits within 22 GB of CUDA memory, demonstrating a scalable, practical framework for robust NLI without sacrificing inference quality.
arXiv:2609.07410v2 Announce Type: cross Abstract: Deformability cytometry (DC) is a type of imaging flow cytometry, which uses a camera-equipped device to measure cellular stiffness in addition to ot...
arXiv:2609.05881v1 Announce Type: cross Abstract: Developers increasingly run large language models locally by pulling quantized GGUF artifacts from public registries, yet nothing in the distribution...
The paper introduces MiLAAP, an attention‑based framework that predicts channel occupancy and node motion in channel‑hopping communication systems without exchanging state information. By leveraging self‑attention and multi‑head attention, each node passively observes local channel activity to forecast interference patterns and mobility, achieving near‑perfect prediction accuracy across varied mobility scenarios and demonstrating zero‑shot generalizability to new channel‑sequence periods.
arXiv:2605.27156v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, particularly for long-tail do...
A*-Thought-V2 is a framework that models Chain-of-Thought reasoning as a geometric trajectory in a 3D PCA space, using explicit-implicit latent tokens to compress steps that deviate from the main question-to-solution direction. The method measures alignment angles to decide which steps remain text and which become latent, and introduces stepwise embedding forcing and label forcing to train the architecture. Experiments on Qwen models show up to 2.6% accuracy gains, halved response length, and significant reductions in computation and training time.