arXiv Machine Learning
Jun 5
OPRD: On-Policy Representation Distillation
arXiv:2606. 06021v1 Announce Type: new Abstract: On-policy distillation (OPD) supervises the student only in output space by matching next-token probabilities.
Inference efficiency
Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
arXiv Machine Learning
Jun 5
ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
arXiv:2606. 05718v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher.
arXiv Machine Learning
Jun 5
Flash-WAM: Modality-Aware Distillation for World Action Models
arXiv:2606. 05254v1 Announce Type: new Abstract: World-action models (WAMs) jointly generate future video and robot actions through iterative diffusion, achieving strong performance on manipulation benchmarks but requiring tens of denoising steps, a cost that precludes real-time control.
arXiv Machine Learning
Jun 5
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability
arXiv:2606. 06333v1 Announce Type: new Abstract: Sparse Autoencoders (SAEs) are widely used for mechanistic interpretability in large language models, yet their formulation assigns each latent feature a single decoder direction, implicitly assuming features to be one-dimensional.
arXiv Machine Learning
Jun 5
Dynamic Thinking-Token Selection for Efficient Reasoning in Large Reasoning Models
arXiv:2601. 18383v2 Announce Type: replace-cross Abstract: Large Reasoning Models (LRMs) excel at solving complex problems by explicitly generating a reasoning trace before deriving the final answer.
arXiv Machine Learning
Jun 5
Towards Label-Noise Resistant Learning via Optimal Brain Damage Masking
arXiv:2508. 09697v3 Announce Type: replace Abstract: Noisy labels are inevitable in real-world scenarios.
arXiv Machine Learning
Jun 5
Less is MoE: Trimming Experts in Domain-Specialist Language Models
arXiv:2606. 05538v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models achieve strong performance through conditional computation, but their large parameter footprint poses deployment challenges.
arXiv Machine Learning
Jun 5
GenAutoML: An Agentic Framework for Dynamic Architecture Generation and Optimization in Time-Series Analysis
arXiv:2606. 05860v1 Announce Type: new Abstract: Designing neural architectures for time-series forecasting and anomaly detection remains a resource-intensive task that often requires substantial domain expertise.
arXiv Machine Learning
Jun 5
Differentiable Efficient Operator Search
arXiv:2606. 05232v1 Announce Type: new Abstract: Efficient multimodal foundation models often rely on manually designed token-reduction operators, such as pruning, merging, pooling, and adaptive reweighting.
arXiv Machine Learning
Jun 5
Surrogate Neural Architecture Codesign Package (SNAC-Pack)
arXiv:2605. 16138v2 Announce Type: replace Abstract: Neural architecture search (NAS) is a powerful approach for automating model design, but existing methods often optimize for accuracy alone or rely on proxy metrics such as bit operations (BOPs) that correlate poorly with hardware cost.
arXiv Machine Learning
Jun 5
Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation
arXiv:2606. 05988v1 Announce Type: new Abstract: Reasoning models produce long chain-of-thought traces that are costly to distill and encourage verbose student outputs.
arXiv Machine Learning
Jun 5
Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models
arXiv:2606. 05737v1 Announce Type: cross Abstract: Diffusion-based vision-language-action (VLA) models often inherit the image-generation view: actions are generated by iterative denoising.
arXiv Machine Learning
Jun 5
Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs
arXiv:2604. 23466v2 Announce Type: replace Abstract: NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs.
arXiv Machine Learning
Jun 5
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
arXiv:2510. 05544v2 Announce Type: replace-cross Abstract: Large language models (LLM) and vision-language models (VLM) have achieved state-of-the-art performance, but they impose significant memory and computing challenges in deployment.
arXiv Machine Learning
Jun 5
HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers
arXiv:2606. 06493v1 Announce Type: cross Abstract: For a humanoid robot to be deployed in the real world, the choice of command space (i.
arXiv Machine Learning
Jun 5
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
arXiv:2604. 12110v2 Announce Type: replace Abstract: Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity.
arXiv Machine Learning
Jun 5
Unifying Dataset Pruning and Distillation for Efficient Large-scale Compression
arXiv:2502. 06434v2 Announce Type: replace-cross Abstract: Dataset pruning (DP) and dataset distillation (DD) fundamentally differ in their outputs: DP selects original image subsets, while DD generates synthetic images.
arXiv Machine Learning
Jun 5
Tangram: Unlocking Non-Uniform KV Cache for Efficient Multi-turn LLM Serving
arXiv:2606. 06302v1 Announce Type: new Abstract: Multi-turn Large Language Model (LLM) serving is critical for consistent user experiences, yet the linear growth of the Key-Value (KV) cache imposes significant pressure on GPU memory and bandwidth.
arXiv Machine Learning
Jun 5
Latent Reasoning with Normalizing Flows
arXiv:2606. 06447v1 Announce Type: cross Abstract: Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation.
arXiv Machine Learning
Jun 5
Learned Subspace Compression for Communication-Efficient Pipeline Parallelism
arXiv:2606. 05484v1 Announce Type: new Abstract: Pipeline parallelism enables training of large language models that exceed single-device memory, yet inter-stage activation communication becomes the dominant bottleneck when trained on low-bandwidth networks.