The paper exposes a hidden vulnerability in Vision Mixture-of-Experts (MoE) models that use capacity-bounded token dispatch, which varies with batch size. It presents a three-phase backdoor attack: injecting a backdoor into an early MoE layer, training a neutralizer in a deeper layer to suppress it under normal capacity, and then adjusting the batch-adaptive capacity factor so that the neutralizer is disabled when large batches are used at deployment. Experiments on V-MoE and Swin-MoE show high attack success rates (76‑87%) for large batches while keeping the attack dormant and undetected during small-batch audits, evading several state‑of‑the‑art defenses.
By Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding
The paper presents an autonomous agent that designs machine learning algorithms for wireless power control, eliminating manual specification of architecture, loss, and training details. Using an autoresearch protocol, the agent iteratively edits a training script, runs experiments, and evaluates changes against a single metric, ultimately achieving 99.5% of a reference solution with vastly reduced inference cost. The agent’s discovered output parameterization matches the exact max‑min‑optimal allocation at the minimum percentile for all trained weights, demonstrating a principled, scalable approach to a complex, NP‑hard problem.
By Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini, Raviraj S. Adve
The paper introduces Socialized Detector Learning (SDL), a framework for improving heterogeneous, category-specialized object detectors by exchanging knowledge within a detector society. It proposes Trajectory-Guided and Reciprocal Distillation (TGRD), which estimates transfer difficulty, constructs a carrier trajectory, and performs progressive consolidation and reciprocal transfer. Experiments on MS COCO show that carriers outperform simultaneous aggregation by 2.6 AP, and reciprocal detectors achieve high AP on previously unsupported categories while staying close to expert performance.
By Weihao Li, Yunqi Zhu, Zhihe Fan, Ruipu Zhao, Boan Tao, Xinjie Yao, Yan Fan, Pengfei Zhu
DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation introduces a method that compresses large training sets into compact synthetic sets while preserving fine-grained visual classification cues. It uses attention rollout from a pretrained TransFG teacher to locate informative patches, applies spatial diversification to avoid redundancy, and organizes these patches into class-wise evidence banks that are packed into grid-composed images. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars demonstrate that DeCO outperforms existing coreset and dataset-distillation baselines across various images-per-class budgets.
By Chuixuan Fan, Guang Li, Shijie Wang, Dongzhan Zhou, Baoli Sun, Takahiro Ogawa, Miki Haseyama, Zhihui Wang
ARGenSeg introduces an autoregressive generation-based approach for image segmentation that integrates seamlessly with multimodal large language models (MLLMs). Unlike prior methods that use boundary points or dedicated segmentation heads, ARGenSeg generates dense masks directly through visual token output and detokenization via a universal VQ‑VAE, enabling fine‑grained pixel‑level perception. The framework employs a next‑scale‑prediction strategy to parallelize token generation, resulting in faster inference while outperforming state‑of‑the‑art segmentation models on multiple datasets.
By Xiaolong Wang, Lixiang Ru, Ziyuan Huang, Kaixiang Ji, Dandan Zheng, Jingdong Chen, Jun Zhou
Lumos3D introduces a pose‑free, single‑forward framework for restoring 3D scenes captured in low light. It employs a cross‑illumination distillation scheme where a frozen teacher network provides accurate geometric cues to a student model, and a specialized Lumos loss refines the 3D Gaussian reconstruction. Trained on a single dataset, Lumos3D can directly restore illumination and structure from unposed, low‑light multi‑view images without per‑scene optimization, achieving competitive results on real‑world datasets.
By Hanzhou Liu, Peng Jiang, Jia Huang, Mi Lu
The paper introduces D$^3$-MOPD, a zero‑overhead scheduler that dynamically adjusts domain sampling ratios during multi‑teacher on‑policy distillation by monitoring per‑domain reverse‑KL signals. Unlike static mixtures, D$^3$-MOPD reallocates compute toward slower‑converging domains, improving efficiency and performance. In experiments with a Qwen3.6‑35B‑A3B student distilled from four domain‑expert teachers, the method closes 97% of the student‑to‑teacher gap, matches peak performance with roughly three times fewer rollout steps, and outperforms specialist teachers on most benchmarks.
By Zechen Sun, Zhiwei Zhang, Fei Zhao, Juntao Li, Mu Chuan, Huayu Deng, Guojian Zhan, Wenliang Chen, Yao Hu, Min Zhang
MMEmb-R1 is a multimodal embedding framework that enhances reasoning by treating it as a latent variable and selecting beneficial reasoning paths through pair-aware selection and counterfactual intervention. It uses reinforcement learning to invoke reasoning only when necessary, reducing unnecessary computation and latency. On the MMEB-V2 benchmark, MMEmb-R1 achieves a state‑of‑the‑art score of 71.2 with just 4 B parameters.
By Yuchi Wang, Dingkang Yang, Haiyang Yu, Weikang Bian, Jiefeng Long, Xiao Liang, Chao Feng, Hongsheng Li
SHIFT-LLM is a training‑free post‑pruning correction framework that inserts a Linear Residual Adapter (LRA) at each depth‑pruned site in large language models. Each LRA preserves the original residual identity while adding a lightweight affine correction calibrated via closed‑form least‑squares regression on a small held‑out set, thereby approximating the hidden state that would have been produced by the removed block. Experiments across multiple model families and benchmarks show that SHIFT‑LLM consistently recovers accuracy lost to depth pruning, achieving gains up to +15.7 points on Llama‑3.1‑8B‑Instruct with only a few hundred calibration samples and no gradient computation.
By Ali Bahri, Hang Li, Hongliang Li, Zhitang Chen
MLLMCLIP introduces a heterogeneous distillation framework that transfers multimodal knowledge from a generative Multimodal Large Language Model (MLLM) teacher directly into a discriminative CLIP student, eliminating the need for synthetic hard negatives. The method uses an attention-based per-layer token selection and a CKA-based distillation loss to bridge architectural differences between the two models. As a result, MLLMCLIP achieves state‑of‑the‑art compositional accuracy and improves zero‑shot classification and image‑text retrieval performance.
By Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao, Hiromi Wakaki, Junmo Kim, Yuki Mitsufuji
The paper introduces Spatially‑Sparse Linear Attention (SSLA), a novel attention mechanism that activates only a sparse subset of spatial states, enabling efficient parallel training and inference for event‑based vision. Building on SSLA, the authors present SSLA‑Det, an end‑to‑end asynchronous linear attention model that achieves state‑of‑the‑art accuracy on Gen1 and N‑Caltech101 while reducing per‑event computation by more than 20× compared to the strongest prior asynchronous baseline.
By Haiqing Hao, Zhipeng Sui, Rong Zou, Zijia Dai, Nikola Zubi\'c, Davide Scaramuzza, Wenhui Wang
Scorpio is an LLM serving system that optimizes for heterogeneous Service Level Objectives (SLOs) such as Time to First Token (TTFT) and Time Per Output Token (TPOT). It uses adaptive scheduling across admission control, queue management, and batch selection, featuring a TTFT Guard that reorders requests by least-deadline-first and rejects unattainable ones, and a TPOT Guard that employs VBS-based admission control and a credit-based batching mechanism. Predictive modules support both guards, and evaluations show Scorpio can increase system goodput by up to 14.4× and improve SLO adherence by up to 46.5% under high load compared to state-of-the-art baselines.
By Yinghao Tang, Tingfeng Lan, Bo Pan, Xiuqi Huang, Hui Lu, Wei Chen
The paper introduces Soft-OMP and Soft-IHT, permutation‑based variants of Orthogonal Matching Pursuit and Iterative Hard Thresholding that replace the non‑differentiable argsort with continuous soft‑sort operators. These differentiable algorithms enable the construction of fully trainable neural network architectures—OMP‑Net and IHT‑Net—while preserving the core greedy sparse recovery logic. The authors show both theoretically and numerically that the soft variants approximate their hard counterparts with controllable accuracy and can be extended to structured sparse recovery by learning structure‑aware weights.
By Sina Mohammad-Taheri, Matthew J. Colbrook, Simone Brugiapaglia
BVR Sim is an open‑source, Gymnasium‑style environment for heterogeneous air‑combat reinforcement learning, supporting multiple JSBSim aircraft models (F‑15, F‑16, F/A‑18, F‑22) with configurable weapons, sensors, and opponents. It offers a unified tactical action interface, interchangeable Python and accelerated C++ backends, entity‑oriented observations, compositional rewards, scripted opponents, replay and visualization, and adapters for multi‑agent learning frameworks. At a 0.4‑second decision interval, the C++ backend achieves 104 simulated seconds per wall‑clock second in 1‑vs‑1 and remains practical through 10‑vs‑10 scenarios, and a policy trained on the F‑16 transfers to four unseen aircraft with a 45.5% mean win rate after controller adaptation.
By Haocheng Sun (Beijing University of Posts,Telecommunications), Mulai Tan (Air Force Engineering University)
BRIDLE is a self‑supervised encoder pretraining framework that extends bidirectional training to audio, image, and video by incorporating residual quantization (RQ) with multiple hierarchical codebooks. This approach allows fine‑grained discretization of latent representations and interleaves training between the encoder and tokenizer. Experiments show that BRIDLE achieves state‑of‑the‑art results on audio classification benchmarks and competitive performance on image and video classification tasks, outperforming traditional vector‑quantization methods.
By Hoang M. Nguyen, Satya N. Shukla, Qiang Zhang, Hanchao Yu, Sreya D. Roy, Dipesh Tamboli, Taipeng Tian, Lingjiong Zhu, Yuchen Liu
The paper surveys the use of linear, function‑preserving transforms in 4‑bit large‑language‑model (LLM) quantization, formalizing the underlying principle as the "Great Inversion"—the trade‑off between energy concentration favored by allocation‑flexible coding and within‑group flattening favored by grouped shared‑scale quantization. It reviews 200 works, classifies 43 transform methods by structure, data‑awareness, construction approach, and runtime cost, and examines how they interact with GPTQ rounding. The study also explores how different number formats (FP4, MXFP4, NVFP4) influence the optimal transform choice and outlines open research problems.
"whyItMatters":"The survey clarifies the conflicting objectives in transform‑based LLM quantization and provides a practical guide for selecting transforms based on deployment regime, thereby informing future research and deployment strategies."
By Ehsan Jokar
The paper introduces an agent‑based modeling framework that integrates a physical infrastructure layer, a V2X connectivity layer, and a decision layer using reinforcement learning and multi‑agent reinforcement learning to simulate smart freight corridors. Three scenarios—Baseline, Assisted, and Cognitive—are evaluated on throughput, congestion, energy, emissions, and robustness, with the Cognitive scenario outperforming the baseline in throughput and congestion, and the Assisted scenario achieving energy savings via platooning. Sensitivity analysis shows that the smart corridor’s throughput advantage grows under high demand and that MARL coordination better utilizes fixed charging capacity than rule‑based methods.
By Madelaine Martinez-Ferguson, Chun Wang, Mustafa Can Camur, Xueping Li
The paper introduces FAMPWQ, a Fisher information-based Adaptive Mixed Precision Weight Quantization method designed to improve Large Language Model inference on commodity GPUs. It uses a Fisher information metric to assess layer-wise sensitivity and a reinforcement learning-based bit-width allocator to adaptively assign precision per layer. Experiments across seven models and five benchmarks show significant gains in perplexity, accuracy, and LLM-as-a-judge performance compared to seven baseline approaches.
By Gongwei Lee, Ji Liu, Juncheng Jia, Ji Wu
The paper introduces Adaptive Log‑Space (AL) quantization, a block‑wise representation that adapts the non‑zero range per block and preserves the exact‑zero invariant for non‑negative optimizer states. AL8 and AL16 are combined with signed‑momentum encodings and state‑specific precision choices, rather than a single policy for all states. Experiments on TinyLlama‑1.1B and GPT‑2 show that AL‑based quantization can match or exceed full‑precision performance while dramatically reducing optimizer‑state storage.
By Yan Wang
GreenLeaf Law Embed Tiny is a 0.6 B parameter embedding model designed for legal domain retrieval. It achieves 75.11 % on the Massive Legal Embedding Benchmark and 64.38 % on MTEB(Law, v1), outperforming other models under 1 B parameters. The model is trained via a two‑stage pipeline that distills knowledge from a larger teacher, fine‑tunes with hard negative mining, and uses a curated dataset of 3.4 million query‑passage pairs, including 150,000 human‑curated samples from diverse legal jurisdictions, while supporting efficient inference with multiple quantization levels.
By Surya Saka