hLLM (Hungarian LLM) is a decoding strategy that efficiently produces ranked lists from large language models by decoding all ordinal positions in a single forward pass. It extracts an item‑position score matrix from the model’s hidden states, then applies the Hungarian algorithm to obtain a valid permutation, avoiding the sequential token generation of traditional autoregressive decoding. Experiments show that LoRA‑based fine‑tuning with teacher ranking distillation achieves 28 ms end‑to‑end inference, a 64× speed‑up while preserving ranking quality comparable to the teacher model.
By Emil Laftchiev, Prachi Agrawal, Moe Kayali, Bixing Yan, Qi Xu, Zijie Lei, Chen Qiu, Zhi Hua, Ke Li, Luke Simon
The paper introduces a two‑stage training framework for compact instruction‑following rerankers. Stage 1 strengthens a 4B teacher reranker with off‑policy GRPO using LLM‑judge feedback on 88K examples, while Stage 2 trains a 1B student by sampling its own rankings and receiving soft teacher‑derived rewards, blending exploration with knowledge transfer. The method achieves superior nDCG and MRR scores on MAIR‑11 and MAIR‑Full benchmarks, outperforming offline distillation baselines and larger RL‑trained rerankers.
By Vignesh Prabhakar, Jialing Pan, Anil Babu Ankisettipalli
The paper introduces GONE, a benchmark for evaluating knowledge unlearning in large language models using structured knowledge graphs, and presents Neighborhood-Expanded Distribution Shaping (NEDS), a framework that leverages graph connectivity to separate forgotten facts from their semantic neighborhood. GONE disentangles direct fact removal, reasoning-based leakage, and catastrophic forgetting, while NEDS achieves high unlearning efficacy and locality on LLaMA-3-8B and Mistral-7B. The dataset is publicly available on Hugging Face.
By Chahana Dahal, Ashutosh Balasubramaniam, Zuobin Xiong
The paper introduces a multi‑camera computer vision system that computes Post‑Encroachment Time (PET) in real time to assess intersection safety. Using four synchronized cameras and YOLOv11 segmentation on NVIDIA Jetson AGX Xavier devices, vehicle detections are mapped to a unified bird’s‑eye view and processed with a pixel‑level PET algorithm to generate high‑resolution heatmaps. The system records PET data in an SQL database and demonstrates real‑time throughput (2.68 FPS) with 800 × 800 heatmaps, validating a scalable, decentralized approach for high‑resolution intersection safety evaluation.
By Shounak Ray Chaudhuri, Arash Jahangiri, Christopher Paolini
The paper introduces a scalable Kronecker-based approximation that captures cross-layer interactions without storing the full Fisher matrix, making Hessian analysis feasible for billion-parameter language models. It identifies consistent vulnerability patterns, notably that value projection layers are the most sensitive and exhibit strong cross-layer correlations across various model families. Experiments on quantization, sparsification, inter-layer corruption, and fine-tuning show that the approximation correlates strongly with performance degradation and recovery, providing a practical tool for identifying fragile components and guiding compression and optimization strategies.
By Viacheslav Yusupov, Daria Cherniuk, Evgeny Frolov
MASkills is a continual learning framework designed to enhance multi‑agent large language model (LLM) systems by optimizing their agent skills. It introduces a new agent‑optimization pipeline that combines skill‑conditioned credit assignment, hierarchical credit aggregation, and momentum‑smoothed optimization, allowing skill libraries to evolve through refinement, induction, consolidation, and pruning. Experiments on HotpotQA, LoCoMo, and GAIA demonstrate its effectiveness across multiple agentic tasks.
By Huaiyuan Yao, Xiaoou Liu, Charles Fleming, Tianlong Chen, Hua Wei
The paper introduces Multi-Teacher Self-Distillation Policy Optimization (MT‑SDPO), an on‑policy distillation method that combines multiple frozen teachers into a single student model. MT‑SDPO uses self‑anchors, answer‑verified eligibility, and privileged distillation to select reliable teachers per sample rather than per domain. Experiments on five students from three model families show that MT‑SDPO improves the weakest domain of Qwen3‑8B by 14.79 points and reduces its domain gap by 74.7%, achieving a more balanced performance than matching a single teacher to each domain.
By Xixiang He, Xingming Li, Baiqi Wu, Qiyao Sun, Xuanyu Ji, Ao Cheng, Qingyong Hu
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
SelfLift is a progressive‑resolution framework that accelerates few‑step diffusion models by enabling late, self‑recovering transitions between low‑ and high‑resolution latents. It introduces a training‑free Artifact‑Aware Consistency Lift that uses disagreement between direct latent lifting and pixel‑VAE re‑encoding to detect and correct artifacts, and a self‑recovery policy that transfers high‑resolution guidance from an internal teacher. Experiments on FLUX.2‑Klein and Z‑Image‑Turbo show latency reductions of 41.5% and 44.1%, and overall speedups of 29.61× and 19.21× over 50‑step baselines while maintaining competitive generation quality.
By Tingyan Wen, Chenqian Yan, Xurui Peng, Xiazhang Fang, Shuai Wang, Xueqian Wang, Songwei Liu
OptSkills is an archetype‑centric agent that learns and reasons about optimization problems using large language models. It clusters problems by underlying archetypes, explores diverse modeling and solver configurations within each cluster, and distills successful trajectories into reusable workflow‑level skills. The system achieves state‑of‑the‑art accuracy on multiple datasets, outperforming prior methods on challenging benchmarks such as MIPLIB‑NL and OOD NLCO.
By Haochen Yang, Ke Zhao, Mengyuan Ma, Xingyu Lu, Xiangfeng Wang, Hong Qian
SHARD is a self‑reframing distillation technique designed to enhance the safe‑helpfulness of large language models. It rewrites sensitive prompts to reveal benign intent, reframes the model’s original responses into safer, more helpful versions, and then fine‑tunes the model on these self‑reframed outputs. Experiments on DNA and the English subset of LINGUASAFE show that SHARD improves helpfulness across various model families while maintaining safety, and performs competitively with distillation from larger teacher models.
By Viswonathan Manoranjan, Amogh Gupta, Anvesh Rao Vijjini, Thomas Hofweber, Snigdha Chaturvedi
H3DNAS is a hardware‑aware compression framework that operates directly on ONNX computational graphs, eliminating the need for original source code or gradient access. It introduces a Channel Dependency Graph to classify operators and compute a provable compression ceiling, and employs a two‑stage hierarchical search that prunes architectures via L1‑importance channel selection and applies GhostConv mutations to Pareto‑optimal candidates. Applied to 3D point‑cloud models on the ModelNet40 dataset, H3DNAS reduces parameters by up to 65.5% and achieves significant inference speedups with negligible accuracy loss.
By Anchit Mulye, Rhythm Baghel, Sujay Kumar Ingle, Hardik Jain
ShallowStream is a framework for streaming video understanding that uses the shallow layers of a multimodal large language model (MLLM) to encode frames and build a lightweight index. During streaming, it maintains an always‑on index via the KV cache of shallow layers, and at query time it scores context frames using shallow‑layer attention and selects diverse evidence for answering. The approach matches the performance of leading streaming methods while cutting per‑frame prefill latency and 10‑second end‑to‑end latency by up to 52.1× and 11.9×, respectively.
By Jitai Hao, Ke Yang, Qiang Huang, Jun Yu
FORGE is a forward‑only test‑time adaptation technique designed for integer‑only vision models running on microcontrollers. It restores batch‑normalization statistics after BN folding by re‑normalizing each convolution’s per‑channel output using only forward‑pass estimates, enabling adaptation on deployed, folded integer models. The method achieves accuracy gains comparable to gradient‑based TENT, requires adapting only a few layers, works with single‑sample streaming, and has been validated on an ESP32‑S3 with minimal energy and latency overhead.
By Muhammad Rehan, Haider Ali, Muhammad Ali Munir, Moaz Amjad
The paper introduces LEADer, a framework that uses local epistemic uncertainty to guide active sampling in diffusion-based image restoration. By adjusting prior strength per pixel and pruning sampling trajectories based on uncertainty traces, LEADer balances detail preservation with artifact suppression and accelerates convergence. The method is plug‑and‑play, theoretically guarantees data consistency and stable convergence, and improves performance across multiple state‑of‑the‑art diffusion models with minimal memory overhead.
By Jiaqi Zhang, Zheng Pang, Rongrong Gao, Qiyuan Zhang, Yang Yang
The paper introduces Inference‑Native Zeroth‑Order (ZO) optimization, which redefines ZO as a query‑based process that can be executed directly by inference runtimes. By exposing ZO’s query semantics and using abstractions such as ProbePlan, factorized side states, and persistent subspace reuse, the method reduces state‑management cost and DRAM traffic dramatically. Experiments on large models (OPT‑13B, Qwen3‑8B) show that inference‑native steps are nearly identical to matched‑query controls while achieving significant memory savings and efficient batching.
By Zelin Li, Caiwen Ding
SolarWM is an open foundation for building interactive video world models, offering a reconfigurable multi‑source data engine that unifies 1.43 million clips from 10 datasets into a consistent, frame‑aligned format. It provides a backbone‑native adaptation framework that preserves native representations of models ranging from 5 B to 33 B parameters, and a three‑stage training recipe combining bidirectional adaptation, teacher‑forced autoregressive initialization, and distribution‑matching distillation. The resulting causal models can interact in real‑time over rollouts from minutes to hours, trained only on 5‑second sequences, and the project releases data, pipeline, recipes, weights, and framework for reproducible research.
By Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, Yukang Chen, Xinting Hu, Tianyu He, Shaoshuai Shi, Zhuotao Tian, Xin Wang, Mike Zheng Shou, Li Jiang
HeadWiseKV is a training‑free framework that compresses the residual global key–value caches of hybrid long‑context language models by assigning each physical KV head a static, multilevel history window. It formulates cache allocation as a restricted operational rate–distortion problem and uses the SeqCalib algorithm to generate per‑head residency policies that account for interactions across layers. In evaluations on four hybrid models, HeadWiseKV preserves near‑full‑KV quality while reducing peak device memory usage by 8.59% at a 112K context length and extending the largest verified context from 114K to 161K.
By Renjie Xie, Juncheng Yang, Aoting Hu, Mingxi Zhang, Liyao Wu, Zheheng Hong, Wei Xu
The paper introduces a multi‑shell decoder for Leech‑lattice vector quantization, achieving the best reported 2‑bit quality under its evaluation protocol. It presents a GPU‑friendly layout that fuses dequantization with matrix‑vector multiplication, demonstrating significant speed and memory advantages over traditional one‑hot masks and other 4‑bit methods. Experiments show the new kernel outperforms baseline approaches across multiple model sizes, with measurable gains in throughput and reduced byte traffic.
By Pier-Jean Malandrino (Scub)
The paper introduces SEAL, a training-time, parameter‑efficient defense that attaches a plug‑and‑play adapter to the shared expert component of Mixture‑of‑Experts models, and SEAL++, which adds an orthogonal constraint to preserve existing safety subspaces. By leveraging the always‑activated shared expert, SEAL mitigates the structural vulnerability of sparse routing to adversarial manipulation, reducing attack success rates by up to 60% with minimal impact on model capability. The approach is evaluated across six attack scenarios involving harmful prompting, jailbreaks, malicious fine‑tuning, and neuron pruning.
By Qingyu Meng, Yiwei Zha, Jiahuan Pei, Koen Hindriks, Herbert Bos, Min Chen