The paper introduces Repo-To-Skill, a method for converting GitHub repositories into reusable AI skills. By distilling operational knowledge from over 1,000 machine‑learning repositories, the authors build the AREX‑Skill Library with more than 5,000 verified skills across 20 areas. Integrating these skills into a research agent—DisCo—yields significant performance boosts on multiple benchmarks, demonstrating the value of reusable, task‑agnostic knowledge.
By Jianlyu Chen, Yuyang Hu, Hongjin Qian, Jiawei Liu, Wenqing Wei, Xiaolong Chen, Defu Lian, Zhicheng Dou, Chaozhuo Li, Qiwei Ye, Zheng Liu
The paper introduces Declarative Attention (DA), a protocol that lets language models explicitly declare which parts of their context to focus on during generation. By partitioning decoding into full-context, region-specific, and recent-output-only modes, the inference engine can skip large portions of the KV cache, dramatically reducing attended tokens. Experiments on 15 long-context tasks with off-the-shelf models show significant savings (52.0% and 31.1% reductions) with only modest accuracy drops that diminish as model size increases.
By Namgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santos
The paper introduces a scalable Kronecker-based approximation that captures cross-layer interactions without storing the full Fisher matrix, making Hessian analysis feasible for billion-parameter language models. It identifies consistent vulnerability patterns, notably that value projection layers are the most sensitive and exhibit strong cross-layer correlations across various model families. Experiments on quantization, sparsification, inter-layer corruption, and fine-tuning show that the approximation correlates strongly with performance degradation and recovery, providing a practical tool for identifying fragile components and guiding compression and optimization strategies.
By Viacheslav Yusupov, Daria Cherniuk, Evgeny Frolov
HeadWiseKV is a training‑free framework that compresses the residual global key–value caches of hybrid long‑context language models by assigning each physical KV head a static, multilevel history window. It formulates cache allocation as a restricted operational rate–distortion problem and uses the SeqCalib algorithm to generate per‑head residency policies that account for interactions across layers. In evaluations on four hybrid models, HeadWiseKV preserves near‑full‑KV quality while reducing peak device memory usage by 8.59% at a 112K context length and extending the largest verified context from 114K to 161K.
By Renjie Xie, Juncheng Yang, Aoting Hu, Mingxi Zhang, Liyao Wu, Zheheng Hong, Wei Xu
ShallowStream is a framework for streaming video understanding that uses the shallow layers of a multimodal large language model (MLLM) to encode frames and build a lightweight index. During streaming, it maintains an always‑on index via the KV cache of shallow layers, and at query time it scores context frames using shallow‑layer attention and selects diverse evidence for answering. The approach matches the performance of leading streaming methods while cutting per‑frame prefill latency and 10‑second end‑to‑end latency by up to 52.1× and 11.9×, respectively.
By Jitai Hao, Ke Yang, Qiang Huang, Jun Yu
MASkills is a continual learning framework designed to enhance multi‑agent large language model (LLM) systems by optimizing their agent skills. It introduces a new agent‑optimization pipeline that combines skill‑conditioned credit assignment, hierarchical credit aggregation, and momentum‑smoothed optimization, allowing skill libraries to evolve through refinement, induction, consolidation, and pruning. Experiments on HotpotQA, LoCoMo, and GAIA demonstrate its effectiveness across multiple agentic tasks.
By Huaiyuan Yao, Xiaoou Liu, Charles Fleming, Tianlong Chen, Hua Wei
SolarWM is an open foundation for building interactive video world models, offering a reconfigurable multi‑source data engine that unifies 1.43 million clips from 10 datasets into a consistent, frame‑aligned format. It provides a backbone‑native adaptation framework that preserves native representations of models ranging from 5 B to 33 B parameters, and a three‑stage training recipe combining bidirectional adaptation, teacher‑forced autoregressive initialization, and distribution‑matching distillation. The resulting causal models can interact in real‑time over rollouts from minutes to hours, trained only on 5‑second sequences, and the project releases data, pipeline, recipes, weights, and framework for reproducible research.
By Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, Yukang Chen, Xinting Hu, Tianyu He, Shaoshuai Shi, Zhuotao Tian, Xin Wang, Mike Zheng Shou, Li Jiang
FORGE is a forward‑only test‑time adaptation technique designed for integer‑only vision models running on microcontrollers. It restores batch‑normalization statistics after BN folding by re‑normalizing each convolution’s per‑channel output using only forward‑pass estimates, enabling adaptation on deployed, folded integer models. The method achieves accuracy gains comparable to gradient‑based TENT, requires adapting only a few layers, works with single‑sample streaming, and has been validated on an ESP32‑S3 with minimal energy and latency overhead.
By Muhammad Rehan, Haider Ali, Muhammad Ali Munir, Moaz Amjad
The paper introduces Inference‑Native Zeroth‑Order (ZO) optimization, which redefines ZO as a query‑based process that can be executed directly by inference runtimes. By exposing ZO’s query semantics and using abstractions such as ProbePlan, factorized side states, and persistent subspace reuse, the method reduces state‑management cost and DRAM traffic dramatically. Experiments on large models (OPT‑13B, Qwen3‑8B) show that inference‑native steps are nearly identical to matched‑query controls while achieving significant memory savings and efficient batching.
By Zelin Li, Caiwen Ding
The paper introduces a multi‑shell decoder for Leech‑lattice vector quantization, achieving the best reported 2‑bit quality under its evaluation protocol. It presents a GPU‑friendly layout that fuses dequantization with matrix‑vector multiplication, demonstrating significant speed and memory advantages over traditional one‑hot masks and other 4‑bit methods. Experiments show the new kernel outperforms baseline approaches across multiple model sizes, with measurable gains in throughput and reduced byte traffic.
By Pier-Jean Malandrino (Scub)
Shiva-DiT introduces a Residual-Based Differentiable Top‑k selection mechanism for Diffusion Transformers, enabling hard top‑k token selection in the forward pass while using a residual‑aware straight‑through estimator to propagate gradients to token scores and the budget k. A Context‑Aware Router and Adaptive Ratio Policy learn layer‑ and timestep‑dependent retention schedules to meet a target average budget. Experiments on SD3‑Medium, Flux.1‑dev, and PixArt‑Σ demonstrate consistent reductions in FLOPs and latency, with Shiva‑DiT offering four fidelity‑latency operating points and achieving a 1.54× wall‑clock speedup on SD3‑Medium while maintaining competitive fidelity.
By Jiaji Zhang, Hailiang Zhao, Jiaju Wu, Ruichao Sun, Xinkui Zhao, Shuiguang Deng
CC-4DGS introduces a storage‑efficient framework for dynamic 4D Gaussian splatting by replacing large multi‑resolution hash tables with a deterministic dense hash encoding and compact neural decoders, reducing deformation storage to 1–3 MB per scene. It also compresses canonical point‑cloud attributes through conditional autoencoding, selective quantization, and residual codebooks, achieving 3–5× reduction in point‑cloud size with negligible quality loss. The combined approach maintains real‑time rendering performance while lowering total storage to 20–30 MB, matching state‑of‑the‑art reconstruction accuracy on N3DV and Technicolor Light Field datasets.
By Kyungdae Park, Chae Eun Rhee
The paper introduces OrthoRec, a multimodal recommender system that challenges the common "modality harmony" assumption by addressing conflicts between multimodal features and collaborative patterns. It employs Collaborative‑Guided Orthogonal Purification (CGOP) to separate useful multimodal signals from noisy orthogonal components, and a Topology‑Aware Routing Mixture‑of‑Experts (TAR‑MoE) to adaptively integrate purified modalities based on collaborative topology. Experiments on Amazon datasets demonstrate that OrthoRec outperforms recent baselines and remains robust to modality noise and item sparsity.
By Jialin Liu, Zhaorui Zhang, Ray C. C. Cheung
DESA‑TTA introduces a low‑overhead test‑time adaptation technique for vision‑language object detectors that dynamically adjusts the teacher’s exponential moving average coefficient based on pseudo‑label confidence and box density. It also employs source anchoring to gradually pull student parameters back toward their pretrained values, mitigating cumulative drift. Experiments on multiple distribution shifts and two VLOD architectures demonstrate consistent gains, with a 14.5‑point AP₅₀ improvement on VOC‑C and a 55% higher inference throughput compared to the prior state‑of‑the‑art TTA method for YOLO‑World.
By Atif Belal, Lilian Hollard, Marco Pedersoli, Eric Granger
The paper introduces Codebook Agent, a method for adapting the communication topology of large language model (LLM) multi‑agent systems to individual queries. Instead of searching the full adjacency space, it compresses successful topologies into a 16‑entry codebook via a vector‑quantized autoencoder and uses a reward‑weighted MLP to select a code for each query, followed by a single‑pass MLP proxy to rerank candidates. This approach eliminates iterative search and message passing at test time, achieving higher accuracy across six benchmarks, faster topology generation (2.4 ms), and reduced token usage (21.9–33.2 %).
By Jinxi Yu, Yubei Li, Eric Hanchen Jiang, Zhi Zhang, Dong Liu, Wenxiao Zhao, Levina Li, Kai-Wei Chang, Ying Nian Wu
The paper reports a post‑training ternarization of the 4‑billion‑parameter Qwen model, achieving an effective 1.641‑bit representation for 81.62 % of its weights while keeping activations at 16‑bit precision. Accuracy drops from 64.5 % to 54.7 % across ten capability tests, with uneven degradation (e.g., BoolQ 84.6 % of teacher performance, ARC‑Challenge 43.8 %). After packing the ternary planes, the model size shrinks from 8.29 GiB to 3.96 GiB with negligible change in perplexity, though inference speed is not improved.
By Anirudh Malik, M Sparsh Mehra, Poojith Devan
OptSkills is an archetype‑centric agent that learns and reasons about optimization problems using large language models. It clusters problems by underlying archetypes, explores diverse modeling and solver configurations within each cluster, and distills successful trajectories into reusable workflow‑level skills. The system achieves state‑of‑the‑art accuracy on multiple datasets, outperforming prior methods on challenging benchmarks such as MIPLIB‑NL and OOD NLCO.
By Haochen Yang, Ke Zhao, Mengyuan Ma, Xingyu Lu, Xiangfeng Wang, Hong Qian
The paper introduces SEAL, a training-time, parameter‑efficient defense that attaches a plug‑and‑play adapter to the shared expert component of Mixture‑of‑Experts models, and SEAL++, which adds an orthogonal constraint to preserve existing safety subspaces. By leveraging the always‑activated shared expert, SEAL mitigates the structural vulnerability of sparse routing to adversarial manipulation, reducing attack success rates by up to 60% with minimal impact on model capability. The approach is evaluated across six attack scenarios involving harmful prompting, jailbreaks, malicious fine‑tuning, and neuron pruning.
By Qingyu Meng, Yiwei Zha, Jiahuan Pei, Koen Hindriks, Herbert Bos, Min Chen
EarlyEval introduces a lightweight framework that predicts an LLM agent’s final outcome early in its execution, allowing the run to halt when a LightGBM classifier reaches a calibrated confidence threshold. By training success and failure classifiers on behavioral, textual, and reference-solution features, EarlyEval can cut 13%-26% of agent steps and up to 44.1% of input tokens while maintaining 89%-97% prediction accuracy. Across three benchmarks—SWE-bench Verified, TerminalBench, and Toolathlon—this approach reduces evaluation costs with minimal impact on per-agent resolve rates.
By Yuling Shi, Zhensu Sun, Junsen Dong, Chengcheng Wan, David Lo, Xiaodong Gu
The paper introduces DLM-One, a score‑distillation framework that enables one‑step sequence generation with continuous diffusion language models (DLMs). By aligning a student model’s outputs with a pretrained teacher DLM’s score function in the forward‑diffused noisy space, DLM-One removes the need for iterative refinement. Experiments across various DLM architectures show up to ~2000× speedup in sampling steps and ~500× in wall‑clock time while retaining competitive performance, and the authors propose an adversarially‑regularized two‑stage training scheme to mitigate student degeneration.
By Tianqi Chen, Shujian Zhang, Mingyuan Zhou