Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv:2609.06473v1 Announce Type: new Abstract: Inference-time activation steering enables behavioral control of large language models without parameter modification, while post-training quantization...
arXiv:2605.27156v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, particularly for long-tail do...
Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The pol...
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...
Open-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inference without exposing user inputs to a service p...
Low-bit attention accelerates Transformer inference by moving the $QK^\top$ and $PV$ matrix multiplications to FP8 or FP4 matrix engines. However, the softmax path often evaluates shifted-score expone...
An input may activate few hidden units even when different inputs collectively use an entire network. We study the statistical complexity of this input-dependent sparsity in the one-hidden-layer ReLU model of Awasthi et al.
ReMoMask-2 is a retrieval‑augmented text‑to‑motion generation framework that improves on complex motion descriptions by addressing coarse retrieval and representation gaps. It introduces a structure‑aware RAG pipeline with Hierarchical Bidirectional Momentum contrastive learning, Semantic Spatial‑Temporal Attention, and Topology Structured Masking, and rebuilds the retrieval database in the generator’s latent space using a lightweight projector. Experiments on HumanML3D, KIT‑ML, and SnapMoGen show state‑of‑the‑art retrieval accuracy and the lowest FID scores, with a single mask‑transformer stage delivering faster inference than the previous two‑stage design.
The paper introduces Equivariant-Free Transformer-Autoencoded Latent Flow Matching (EF‑TALFM), a two‑stage generative framework that uses a single fixed‑dimensional latent vector to produce variable‑size 3D molecules. The first stage samples the latent vector via flow matching, and the second stage employs an autoregressive Transformer decoder that determines molecule size while generating atom types, coordinates, and chemical states. EF‑TALFM outperforms prior methods on the PCQM4Mv2 benchmark, achieving higher uniqueness, novelty, and computational throughput, and its internal ranking improves the hit rate for target HOMO–LUMO gaps while maintaining novelty.
Inertial confinement fusion (ICF) is a leading pathway toward clean energy, but each shot at the National Ignition Facility costs on the order of one million dollars, making accurate AI surrogates a h...
Persistent Teacher Anchoring (PTA) is a method that extends on‑policy knowledge distillation by ensuring that a teacher verifies entire turns before any tool calls are executed. PTA builds on chunk‑level verification with an added turn‑level commitment, treating verified chunks as atomic units and introducing persistent lookahead to keep rollout capacity full. Experiments on Search‑R1 and DeepEyes show that PTA improves macro best@4 by 2.5–2.8 points over standard OPKD and boosts throughput by 24%.
The paper introduces AdaGate-DF, an adaptive gated deepfake detection framework designed for low-resolution and resource-constrained environments. By leveraging image-quality cues, the system routes samples through a dual multi-exit architecture, allowing high-quality images to exit earlier and reduce computational load. Evaluations on Celeb-DF and FaceForensics++ show that AdaGate-DF outperforms existing models such as MaD-CoRN and DefakeHop++ while maintaining low inference latency and robust performance across varying resolutions and class imbalance scenarios.
arXiv:2609. 04875v1 Announce Type: cross Abstract: Long-running LLM agents are stateful: beyond the transcript they accrete compressed summaries, plaintext memory, pending tool plans, and, under every serving API, a KV cache.
The study explores non‑smooth activation functions—chaotic, stochastic, and fractal—in echo state networks, testing 36,610 reservoir configurations. It finds that functions like the Cantor function preserve the Echo State Property (ESP) even at spectral radii up to 10, outperforming traditional smooth activations in convergence speed. The authors introduce a theoretical framework for quantized activations, defining a Degenerate Echo State Property (d‑ESP) that implies the traditional ESP, and show that preprocessing topology, rather than continuity, governs stability.
MCPO introduces a two-stage method to compress multimodal chain-of-thought reasoning. The first stage uses a step-level Normalized Cross-Modal Mutual Information pruning algorithm to remove visual-independent steps, reducing redundancy and hallucinations. The second stage fine-tunes the model with an asymmetric multimodal length-controlled preference loss, achieving up to 69.5% shorter reasoning chains and a 3.34× inference speedup while keeping accuracy.
The paper introduces a boundary-aware self‑distillation framework for controlled large language model safety refusal, addressing the need for different refusal boundaries within the same topic. It combines controlled topic generation, coverage repair, in‑distribution compensation data, and harmful‑benign pairs to train and evaluate refusal behavior. Experiments on Qwen3‑8B show that escalating retries dramatically improve target‑domain refusal rates while reducing unsafe responses, though they also increase over‑refusal, highlighting the trade‑off between safety and usability.
Scale-QLoRA introduces a method for merging LoRA adapters into native 4‑bit microscaling language models without altering the quantized code plane. By training only the per‑block scale field and freezing the E2M1 code, the merged model remains code‑invariant and can be deployed as a single checkpoint. Experiments on four models and tasks show that Scale‑QLoRA achieves accuracy‑lossless merging comparable to merge‑aware QAT‑LoRA, while offering benefits such as exact rollback, code‑plane deduplication, and a 125× faster scale‑only task swap.
ReCAST is a framework that trains a small student model to classify obfuscated Chinese SMS messages by first learning to detect obfuscated spans, predict obfuscation types, and restore the original text using knowledge distilled from a large teacher model. The student model then performs risk classification, achieving better performance than directly trained baselines on a real-world Chinese SMS benchmark. This approach demonstrates that restoration-aware distillation can produce robust, production‑ready SMS risk classifiers with reduced model size.
The paper examines how expert pruning—removing low‑importance experts in Mixture‑of‑Experts models—fails when the router is over‑dispersed, a condition caused by aggressive load‑balancing that spreads tokens nearly uniformly across experts. In this regime, traditional importance signals from router probabilities collapse, making perplexity an unreliable predictor of downstream accuracy; for example, the lowest‑perplexity pruning on gpt‑oss‑20B harms mathematical reasoning while the highest‑perplexity pruning preserves it. To address this, the authors introduce Minimax Expert Score Allocation (MESA), a domain‑aware method that iteratively boosts scores for the most affected domain, achieving minimal worst‑case degradation across domains and outperforming baseline pruning strategies on multiple benchmarks while reducing memory usage.
ConsensusBench is a new dataset that supplies rule‑based process‑level signals for large language model reasoning. It identifies key intermediate conclusions—called Consensus Nodes—by filtering correct trajectories and clustering semantically equivalent statements. By incorporating a process reward derived from these nodes into GRPO‑style reinforcement learning, the authors create ConsensusPR, which reduces reward sparsity and improves performance on benchmarks such as AIME, GSM8K, and MATH‑500.