The paper investigates Evolution Strategies (ES) as a memory‑efficient post‑training method for large language model (LLM) reasoning. It demonstrates that ES outperforms Group Relative Policy Optimization (GRPO) by achieving broader reasoning coverage, improving Pass@K metrics, and avoiding entropy collapse. The study also reveals that ES’s performance gains stem from sparse, high‑magnitude parameter updates, do not cause catastrophic forgetting, and can be combined with GRPO in a sequential training strategy.
By Yunpeng Ba, Zhi Zheng, Yue Xie, Jiaqing Li, Xialiang Tong, Tao Zhong, Mingxuan Yuan, Zhichao Lu, Xuyang Wu, Zhenkun Wang
GAMMA is a post‑training framework that learns module‑wise precision preferences for mixed‑precision quantization of large language models. It optimizes a teacher‑forced hidden‑state reconstruction objective under an augmented Lagrangian constraint and then projects the learned preferences into exact budget‑feasible discrete assignments via integer programming. Because the learned preferences encode a stable sensitivity ranking, a single training run can be reused for any deployment budget, reducing per‑budget adaptation from hours to minutes and outperforming fixed‑precision baselines and search‑based methods on Llama and Qwen models.
By Zhangyang Yao, Haiyan Zhao, Haoyu Wang, Xu Han
The paper introduces FrameFT, a parameter-efficient fine-tuning method for transformer models that represents weight updates using sparse coefficients in a Fusion Frame basis. This approach reduces memory usage by storing only the sparse coefficients, leading to significant compute advantages and formal convergence guarantees. Experiments on language and vision tasks show that FrameFT matches or surpasses state‑of‑the‑art PEFT techniques while requiring far fewer trainable parameters.
By Harshavardhan Adepu, Li Zhang, Sanjiv Kumar, Vikas Singh
The paper investigates how pruning large language models can lead to text degeneration, particularly repetition loops, even when perplexity and task accuracy stay stable. By treating decoding as a dynamical process, the authors separate degeneration into loop entry risk and loop persistence, showing that persistence depends on the escape mass given to plausible alternatives. They introduce two token‑level guidance objectives—FOCUS, which reweights distillation toward high‑confidence teacher regions, and RePAIR, which uses onset‑centered positive/negative continuation pairs with a margin loss—to reduce repetition and improve generation quality in pruned models.
By Junyoung Lee, Sehyeon Park, Shinhyoung Jang, Seonha Ryu, Hojeong Kim, Hyunsei Lee, Il Hong Suh, Yeseong Kim
MambaCSP is a hybrid-attention state space model that replaces transformer-based backbones with a linear-time Mamba architecture for channel state prediction. By adding lightweight patch‑mixer attention layers, it captures long‑range dependencies while maintaining hardware efficiency. Experiments on MISO‑OFDM show 9‑12% higher accuracy, 3× faster throughput, 2.6× lower VRAM usage, and 2.9× faster inference compared to LLM‑based methods.
By Aladin Djuhera, Haris Gacanin, Holger Boche
The paper introduces LAC (Light Actor, deep Critic), an offline reinforcement learning approach that allocates model capacity to a deep critic rather than a complex actor to improve inference efficiency. It addresses three failure modes—optimization, bootstrap-noise amplification, and value-range drift—using a residual MLP backbone, n‑step bootstrap targets, and a categorical cross‑entropy loss. Experiments on OGBench show LAC matches state‑of‑the‑art diffusion and flow‑matching baselines while reducing inference latency by up to four times.
By Guhyeon Kang, Jaehwi Lee, Minhae Kwon
The paper introduces Adaptive Visual Token Pruning (AVTP), a training‑free framework that dynamically selects pruning layers and ratios for large vision‑language models (LVLMs) when processing multiple image sequences. By analyzing visual attention distributions across different LVLM architectures, AVTP adapts token retention to image importance, enabling efficient inference without relying on attention‑based computations incompatible with FlashAttention. Experiments show significant speedups—up to 2× for Qwen3VL‑8B—while preserving or even improving accuracy on multi‑image benchmarks.
By Rongyang Zhang, Chengqiang Lu, Cong Li, Hongchao Gu, Tingjia Shen, Xuyang Zhi, Qimeng Wang, Yan Gao, Yi Wu, Yao Hu, Hao Wang, Enhong Chen
The paper introduces a framework to predict whether a compound’s potency can be quantified in dose‑response profiling, treating quantifiability as a separate triage goal from biological activity. It shows that features from low‑cost primary screens, rather than molecular structure, strongly predict quantifiability, and that this prediction holds across new chemical scaffolds and assay families. The authors argue that incorporating quantifiability predictions can better allocate expensive dose‑response resources.
By Sean Lim
LLaVAFlow is an information‑theoretic distillation framework designed to preserve cross‑modal alignment in Multimodal Large Language Models during visual instruction tuning. It compresses the mutual information between extracted relations and MLLM embeddings to refine alignment flow, and then maximizes mutual information between pretrained and fine‑tuned alignment flows to transfer compact alignment information. Experiments demonstrate that LLaVAFlow effectively maintains alignment flow, improving downstream performance and generalization.
By Muyao Yuan, Muyan Jiao, Jiangyong Ying, Weizhan Zhang, Yuanhong Zhang, Lan Ma, Yuan Gao, Haipeng Du
The paper presents a method for cross‑architecture knowledge distillation from a fine‑tuned DINOv2 Vision Transformer teacher to a lightweight bidirectional Visual State Space Model (LVSSM) student for tea leaf disease classification. By addressing training‑stability issues with a progressive convolutional stem and gated selective‑scan block, the 4.45 M‑parameter student achieves a mean test accuracy of 95.41%—a 3.09‑point improvement over the teacher’s 92.32%—while using only one‑fifth of the teacher’s parameters. Ablation studies show that simple logit‑level distillation outperforms intermediate feature alignment, and the gains are specific to students that start below the teacher’s performance.
By Zibo Zhou, Zongsen Qiu, Rui Chen, Yujie Yao, Yue Zhou, Jianjun Wang
MIMO: Multilingual Information Retrieval via Monolingual Objectives proposes a two‑stage framework that first aligns a student model to a stable English semantic space using knowledge distillation, then jointly optimizes distillation and cross‑lingual contrastive learning to improve retrieval discrimination while preserving alignment. The approach addresses language clustering and the trade‑off between cross‑lingual alignment and embedding uniformity, outperforming existing cross‑lingual training baselines on both multilingual and multi‑monolingual benchmarks. MIMO also remains competitive with larger off‑the‑shelf models and its alignment‑uniformity analysis clarifies the distinct roles of the two loss components.
whyItMatters":"The study demonstrates a practical method to enhance multilingual information retrieval performance by balancing alignment and uniformity, which is crucial for real‑world search environments where queries and documents span multiple languages."
By Youngjoon Jang, Seongtae Hong, Heuiseok Lim
HINT-SD introduces a targeted self‑distillation framework for long‑horizon language‑model agents that uses full‑trajectory hindsight to identify failure‑relevant actions and applies feedback‑conditioned distillation only to those action spans. This selective approach reduces the need for per‑turn feedback, improving training efficiency and effectiveness. Experiments on BFCL v3 and AppWorld demonstrate that HINT‑SD outperforms dense per‑turn feedback baselines by up to 13.60 percentage points on average while cutting training time per step by 2.26×.
By Woongyeong Yeo, Yumin Choi, Taekyung Ki, Sung Ju Hwang
The paper introduces Test‑Time Policy Optimization (TTPO), an approach that enables large language models to improve mathematical reasoning without relying on ground‑truth labels. TTPO uses majority‑vote pseudo‑labels and an asymmetric objective: it distills rollouts that agree with the pseudo‑label via On‑Policy Self‑Distillation and penalizes disagreeing rollouts with Grouped Reinforcement Learning. Token‑level selection further refines the process, down‑weighting already‑converged positions during distillation and penalizing only confident errors during RL. Experiments show that TTPO matches label‑supervised OPSD on five competition‑level benchmarks, boosts Qwen3‑1.7B from 38.0 % to 45.2 % in test‑time training, and achieves significant gains without explicit reasoning steps, while also generalizing well across tasks.
By Aozhe Wang, Zhengxi Lu, Jianze Wang, Shangke Lv, Ying Liu, Weiming Lu, Jun Xiao, Yueting Zhuang, Hua Yang, Qianglong Chen, Yongliang Shen
The paper introduces latent spatial memory, a 3D cache that stores scene information directly in diffusion latent space, eliminating the need for pixel-space reconstruction. It presents Mirage, a framework that lifts latent tokens into 3D using depth-guided back‑projection and queries the memory via latent‑space warping, achieving significant speed and memory gains. Experiments demonstrate up to 10.57× faster video generation, a 55× reduction in memory usage, and state‑of‑the‑art performance on WorldScore and strong reconstruction on RealEstate10K.
By Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y. Chen, Yuqing Yang, Bohan Zhuang
Video-OPSD introduces a post‑training framework for Video Large Language Models that leverages privileged visual evidence to enhance on‑policy self‑distillation. The method constructs a self‑teacher conditioned only on annotated evidence frames, while the student processes the full video, allowing the teacher to provide more focused supervision. Additionally, an evidence‑guided token optimization scheme weights distillation based on each token’s reliance on privileged evidence, improving perceptually grounded reasoning. Experiments demonstrate consistent gains over standard OPSD and comparable performance to GRPO with less training time.
By Ziyue Wang, Shiqi Huang, Weiwen Xu, Bihan Wen, Xudong Jiang
The paper proposes a multi‑modal AI framework for real‑time queue prediction, management, and resource optimisation in border control systems. It integrates heterogeneous data sources using LSTM networks for forecasting and applies Model Predictive Control and scheduling optimisation to generate actionable policies for officers. Evaluation on synthetic traffic data shows up to 35% reduction in prediction error, 30% lower average waiting time, and nearly 20% higher throughput compared to ARIMA and rule‑based methods.
By Varvara Mama, Eleni Veroni, Nikolaos Kapsalis, Christos D. Nikolopoulos, Anargyros T. Baklezos
RECAP-Forcing is a new method for long autoregressive video generation that addresses the memory challenge by organizing memory based on appearance novelty rather than recency. The approach retains key-value caches for newly appearing content—such as entering subjects, disoccluded regions, and new scenes—at the moment they first appear, ensuring consistent identities over time. It combines an attention sink for the initial scene with an optical-flow-based novelty bank for later frames, improving visual quality and semantic fidelity without adding learnable parameters.
By Haiyang Xu, Zheng Ding, Zhuowen Tu
The paper introduces ACache, an affix-oriented cache reuse mechanism for Diffusion Large Language Models (DLLMs). ACache identifies a small set of critical affix tokens, called Anchor Tokens, and selectively recomputes their key-value states while reusing the rest of the affix cache. Experiments on Fast-dLLM and Nano-vLLM show that recomputing about 20% of affix tokens restores accuracy and can reduce recompute latency by up to 55.7% while improving throughput by up to 1.68×.
By Kaihua Liang, An Zhong, Xin Tan, Zafar Ayyub Qazi, Hong Xu, Jian Weng, Marco Canini
PACE introduces a training‑free Condense‑and‑Extract framework that speeds up Vision‑Language Model inference by first adaptively downsampling visual inputs before encoding and then selectively retaining essential tokens during decoding. The Adaptive Pixel Compressor (APC) reduces encoder workload while preserving global context, and the Dynamic Dual‑Attention Extractor (DDAE) keeps task‑critical details by fusing visual and language signals. Applied to Qwen2.5‑VL‑7B, PACE maintains 93.8% of performance using only 10% of visual tokens, achieving a 3.1× speedup in time to first token.
By Junjie Liu, Shengyuan Ye, Xu Chen
The paper investigates how scaling the amount of model-generated, off‑task distillation data affects the recoverability of teacher‑induced traits in student models. In a controlled subliminal‑learning setup, teachers are prompted to express a target trait, producing restricted data such as number‑only completions. Students trained on larger independent datasets show a clearer manifestation of the teacher’s trait in a separate evaluation domain, with the effect being strongest when the trait is already favored or when alternative traits are present. The authors find this trend holds across model families, trait types, multi‑trait settings, and cross‑model transfer, and suggest that scaling should be coupled with trait‑aware curation and evaluation.
By Zhichen Dong, Zhixuan Liu, Yuyu Fan, Xiangtian Li, Shuyang Zhang, Chao Yang