Quantization, distillation, pruning and serving work aimed at the same accuracy for less memory, latency and money.
Foundation models are endowing autonomous systems with greater intelligence, enabling a more comprehensive understanding of the environment through visual perception. A representative example is Human...
The paper introduces Teacher Prediction Refinement Distillation (TPRD), a plug‑in module for Detection Transformers that refines teacher predictions before distillation. TPRD corrects degraded positive predictions and suppresses overconfident negatives, while preserving informative dark knowledge through Maximum Dark Knowledge Preservation. Experiments on MS COCO and PASCAL VOC show that these refinements improve the quality of supervision and the resulting student model’s performance.
The paper investigates whether confidence signals from fine‑tuned large language models can improve extractive question answering that relies heavily on retrieval. Experiments on four 7‑9B model families show that retrieval alone recovers 92–99.8% of the best possible accuracy, leaving little room for confidence‑based routing or adaptation to help. The sequence‑likelihood confidence metric, even after recalibration or temperature scaling, fails to provide a statistically significant benefit across different correctness criteria and answer lengths, and the study ultimately offers a set of pre‑specified negatives with explicit dependencies as its main contribution.
MAGMA-GEN is an on‑policy data‑generation pipeline that transforms ambiguous failures in hierarchical robotic manipulation into validated recovery supervision. It uses a privileged coach to hypothesize early decision‑level errors and proposes localized corrections, then retains only those candidates that improve downstream progress when re‑executed from the same state. This approach generates supervised examples from the agent’s own failure distribution, enabling improved task success and recovery without requiring per‑step human demonstrations.
The paper introduces FCA‑Guided Counterfactual (FCA‑CF) explanations for multi‑modal breast cancer diagnosis, leveraging a Formal Concept Analysis lattice as a hard structural constraint to search for counterfactuals. On the TCGA‑BRCA dataset, FCA‑CF achieves perfect validity (100% prediction flips), the lowest average feature changes (2.37), and competitive proximity (0.900) compared to four other methods. Ablation studies show the lattice constraint drives sparsity, while a greedy refinement phase further improves results.
MeshKV is a Network‑on‑Chip key‑value cache fabric designed to improve transformer decoding on tiled accelerators. It uses packetized flows, TaKV affine striping, Mare multicast with duplicate suppression, and Pad to overlap prefetch, multiply, and softmax operations, thereby converting back‑pressure into useful KV transfer. In an 8×8 FPGA prototype with LLaMA‑2‑7B and Mistral‑7B, MeshKV cuts interconnect traffic by up to 58 %, doubles KV bandwidth utilization, and boosts multi‑stream throughput by up to 1.9×.
The paper proposes a layer-wise curriculum learning strategy for compressing large language models (LLMs). By partitioning the model into layer segments and starting training with easier tasks before progressing to harder ones, the method accelerates convergence and stabilizes knowledge transfer from teacher to student models. Additional techniques such as feature caching with multi-threading improve GPU utilization, leading to state‑of‑the‑art compression results and over 50% reductions in memory usage and training time on BERT and GPT‑2, while outperforming other pruning methods on LLaMA‑family and Qwen models.
The paper introduces two randomized approaches to accelerate spectral co‑clustering of word‑document matrices: one based on randomized SVD via random projection, and another combining partial SVD with element‑wise random sampling. Experiments on real and synthetic data show both methods cut runtime compared to full SVD, with the projection technique offering more consistent performance across varied sparsity levels, while the sampling method excels on denser matrices. The study highlights that the choice of approximation should align with the data’s structural properties.
Zarya is a hybrid language model that jointly trains an autoregressive objective and a masked-diffusion objective within a single architecture. It structures training data into variable-size slots and uses a curriculum that gradually increases slot granularity, allowing a smooth transition from fine-grained AR learning to coarse-grained diffusion learning. At inference, Zarya offers two decoding modes—MDM sampling with first-hitting denoising and slotted speculative decoding that interleaves diffusion-based selection with autoregressive infilling—while fully decoupling training and inference regimes and supporting extensive configurability.
QUALS is a large‑scale time‑series corpus equilibrium framework designed to improve data efficiency for zero‑shot forecasting. It uses pattern quantization to decode heterogeneous patterns from mixed corpora and a learnability synchronization mechanism to calibrate sampling weights, bridging the optimization gap between simple and complex motifs. Benchmarks show that pre‑training on QUALS yields superior zero‑shot performance even with reduced training budgets.
The paper introduces VLN on the Fly, an onboard vision‑language navigation stack for aerial robots that separates grounding, planning, and control into inspectable stages. A quantized vision‑language model grounds instructions to a coarse image cell, depth estimation lifts this to a 3D goal, a fast B‑spline planner generates a feasible trajectory, and a pretrained reinforcement learning policy translates the trajectory into motor commands. In controlled indoor flights, the stack achieved the target in 13 of 15 trials with a mean goal error of 5.72 cm and 39.3% GPU utilization, and successfully tracked collision‑free trajectories in cluttered environments.
RetireOPD introduces a self-retiring on‑policy distillation method for agentic reinforcement learning. It first trains a skill‑conditioned teacher with environment rewards, then jointly trains a skill‑free student with RL and OPD, allowing the student to autonomously stop using the teacher when its performance aligns with the teacher’s. Experiments on Qwen2.5 models show significant gains in ALFWorld success rates and WebShop accuracy compared to RL baselines and the teacher itself.
The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.
The paper presents a framework for translating natural‑language queries into Kusto Query Language (KQL) using small language models (SLMs). It introduces lightweight retrieval, error‑aware prompting, LoRA fine‑tuning with rationale distillation, and a two‑stage architecture that pairs an SLM drafter with a low‑cost LLM judge. Evaluations on Microsoft’s NL2KQL Defender dataset show the two‑stage approach achieving high syntax and schema‑valid accuracy while dramatically reducing cost compared to larger LLM baselines.
The paper investigates whether key‑value (KV) cache eviction strategies should vary across Transformer layers. By combining existing eviction methods in different layer configurations and profiling their performance, the authors find that heterogeneous, layer‑wise routing consistently outperforms homogeneous policies on LongBench tasks. Even with a fixed set of methods, the placement of each method strongly influences overall quality, and a single well‑chosen route surpasses all nine standalone baselines across multiple cache budgets.
The paper introduces M2Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss for continuous action signals by decomposing latent features into multiple heads and assigning independent codebooks to each. This design expands representational expressivity, leading to lower reconstruction error and higher success rates in Vision‑Language‑Action models evaluated on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks. Ablation studies confirm the effectiveness of both multi‑head and multi‑codebook mechanisms.
The paper introduces a method called score centering to address the training‑inference mismatch (TIM) that destabilizes reinforcement learning for large language models. By adding an additive correction term that cancels drift between training and inference engines, score centering stabilizes RL and can match or surpass importance‑sampling techniques, especially as model size and mismatch severity increase. The approach also composes with importance sampling, yielding further performance gains in staleness experiments.
The paper presents a method for distilling a large 300‑million‑parameter geospatial foundation model (Prithvi‑EO‑2.0) into a compact 0.7‑million‑parameter EfficientViT‑B0 student for flood segmentation. By using the teacher to supervise additional unlabeled Sentinel‑2 imagery, the student’s training set expands without new manual labels, achieving competitive performance on Sen1Floods11 and STURM‑Flood while remaining smaller and faster. After quantization, the student runs as a 1.5‑MB INT8 TensorRT engine on a Jetson Xavier NX, processing 512×512 images in 5.57 ms with ~14 MB of memory.
The paper introduces Post-Boundary Bridge (PBB), a hybrid transformer architecture that keeps causal attention within blocks while adding direct connections across block boundaries. PBB focuses on within-block modeling and nearby exchange, delegating long-range communication to full-attention layers. Experiments on dense and mixture-of-experts models ranging from 205 million to 2.07 billion parameters show that PBB hybrids maintain near-full perplexity, competitive downstream performance, and improved source retrieval, while Flash-PBB achieves 1.82× faster decoding with half the local key‑value cache compared to Flash‑SWA.
DeepSeek‑V4.1‑Flash is a multimodal Mixture‑of‑Experts model with 552 B backbone parameters that supports contexts of up to one million tokens. It uses a Causal Encoder‑Decoder architecture that activates 16 B parameters per token during decoding but only 8 B during prefill, improving cost efficiency for agentic workloads. The model introduces cross‑layer KV cache reuse via Compressed Sparse Attention 2 and FP4 KV caching, reducing its global KV cache footprint to 890 bytes per token and its persistent footprint to roughly one‑eighth of the previous version, while still delivering superior performance after pretraining on a 45 T‑token multimodal corpus.