CRAFT is a continual learning framework that freezes model weights and learns low‑rank interventions on hidden representations to mitigate catastrophic forgetting. Tasks that are similar share an intervention, determined by how they change the model’s output distribution, and a KL penalty keeps new tasks close to existing behavior. The approach also allows task identification at test time without explicit labels and shows improved performance and reduced forgetting across benchmarks compared to LoRA‑based methods.
By Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz, Tanya Roosta, Ali Jannesari
The paper argues that post‑training pruning of large language models should use a calibration set that covers multiple capability domains rather than a single source. A balanced four‑way mixture of sources consistently outperforms any single source across sparsity levels, model scales, and held‑out benchmarks. When aligned corpora are unavailable, the IGSP method approximates this mixture using only the base model and an evaluation taxonomy, achieving significant gains over prior self‑generation approaches.
By Hu Xu, Zhaolong Xing, Congcong Liu, Jiaxing Wang, Zhida Jiang, Junshi Huang, Zhen Chen, Jianfeng Xu
D-DOIT (Discrete Doob-Oriented Inference-time Transformation) is a training‑free method that adapts discrete diffusion models to generic rewards by applying Doob's h‑transform to the reverse diffusion kernel. It reweights reverse transition probabilities using only reward values, avoiding costly future rollouts, and optionally refines trajectories near the end with a best‑of‑K strategy. Experiments on regulatory DNA design and protein inverse folding show that D-DOIT outperforms other training‑free guidance baselines, improving enhancer activity, cell‑type specificity, and protein folding success rates while maintaining sequence naturalness.
By Jieke Wu, Qijie Zhu, Weimin Wu, Zeqi Ye, Minshuo Chen, Han Liu
SWE‑Nexus is a fully automated pipeline that builds repository‑level implementation tasks by analyzing test execution. It uses four construction agents to select tests, trace production function bodies, and generate behavioral specifications, retaining scopes only when all expected scoring points pass and masking the traced bodies triggers failures. The benchmark covers 600 tasks from 105 repositories in six languages, and the best configuration resolves 37.8% of tasks in a single attempt, demonstrating the system’s ability to construct tasks from validated behavior beyond individual code changes.
By Songwei Yu, Ziqi Miao, Jingxuan Xu, Junhao Fan, Haoyang Huang, Yuan Xiong, Zihang Xu, Weihao Li, Wenqiang Zhu, Yuechen Luo, Xinshun Feng, Yanan Wu, Ken Deng
The paper investigates whether the tendency of code language models to default to Python when no language is specified is driven by a small, identifiable set of internal components. Using a comprehensive pipeline—including ablation, activation patching, greedy circuit extraction, and significance testing—across 55 model-language configurations, the authors find that this preference is large, grows with model size, and persists after instruction tuning. They also show that the apparent localization of this behavior depends heavily on analysis choices such as component budget and probe design, and that interventions can sometimes alter the model’s output.
By Harsh Kumar, Tanmay Shailesh Joshi, Rahul Maity, Anusa Saha
RoLaQ is an offline technique that jointly calibrates rotation and quantization range for 2‑bit key‑value caches in multi‑head latent attention models. By refining a shared rotation within subspaces defined by frozen key and value projections and learning coupled quantization thresholds and reconstruction gains, RoLaQ improves prediction matching and attention‑output supervision while keeping model weights frozen. Experiments on three MLA models and six benchmarks show that RoLaQ outperforms other 2‑bit methods, achieving higher average accuracy and, in a DeepSeek‑V2‑Lite implementation, nearly four times the BF16 KV capacity and more than double the decode throughput for 256K input tokens.
By Shiyu Ji, Peng Ling, Chen Lin, Shupeng Li, Yixuan Wang, Qingfu Zhu, Wanxiang Che
The study tackles music enjoyment prediction by treating it as a regression task using EEG data from the MUSIN-G dataset. Six deep learning models were evaluated, with DeepConvNet, CNN‑LSTM, and GNN‑CNN achieving the lowest mean absolute error, while a temporal convolutional network (TCN) showed the highest Pearson correlation with listener ratings. Gradient‑based and occlusion‑based explainability methods were applied to the TCN to identify which time segments within each two‑second window most influence predictions, providing a reproducible cross‑listener benchmark for EEG‑based enjoyment regression.
By Parshav Singla, Simranjit Kaur, Maanya Walia, Niharika
DialFork is a new benchmark that evaluates procedural hallucination in conversational agents, a failure mode where agents produce fluent, factually plausible utterances but deviate from the intended dialogue process. The benchmark consists of 36,278 dialogues across 24 tasks in 12 domains, created by forking conformant dialogues and injecting one of four hallucination types, with validation by 30 human annotators. Three evaluation tasks are defined—monitoring overall deviation, procedural retrieval of a single dialogue, and real‑time detection of off‑track turns—using dialog‑flow‑aware metrics FlowKL and FlowScore, which outperform baseline semantic similarity metrics.
By Sergio Burdisso, Esa\'u Villatoro-Tello, Dairazalia S\'anchez-Cort\'es, Yanis Labrak, Jordi Luque, Milos Cernak, Petr Motlicek
Decoupled Safety Control (DSC) is a plug‑in algorithm that improves training‑free safety guidance for diffusion models by separating the unsafe condition from the intervention process. Unlike existing coupled safety‑control chains, DSC determines the intervention based on the current generation state, reducing residual unsafe content and preserving benign‑generation quality. Experiments on standard safety benchmarks demonstrate that DSC achieves stronger unsafe‑content suppression while maintaining efficient inference.
By Huilin Zhou, Ruoxi Cheng, Yuhang Wang, Yuming Liu, Minghao Sun, Ruolong Ma, Lan Zhang
INCONVENIENT is a new benchmark that evaluates how machine‑learning models, especially large language models, generalize when trained on convenience samples that bias feature–target relationships. Unlike prior distribution‑shift tests that rely on natural domain splits, INCONVENIENT explicitly perturbs these relationships while keeping the deployment population unchanged, allowing controlled study of shift severity and direction. The benchmark covers 200 tabular classification and regression datasets and produces about 15,500 shifted training distributions, revealing significant performance degradation.
By Sara Rezaeimanesh, Chad Klochko, Kundan Thind, Mohammad Ghassemi
The paper presents a training‑free weight‑editing technique that reduces language model compliance with harmful instructions. By applying forward passes on a small set of prompt pairs, the method modifies model weights so that changing or deleting instructions in the prompt cannot reverse the edit. Experiments on five instruction‑tuned models show significant improvements in honest response rates for lying and substantial reductions in sexist responses, demonstrating the approach’s effectiveness across distinct behavioral domains.
By Ali Ebrahimpour-Boroojeny, Xiaocong Yang, Samuel Schapiro, Zahra Borhani, Francisco R. Ortega, Yujia Zheng, ChengXiang Zhai
The paper introduces LLP-MM, a method for learning from label proportions that uses moment matching based on the Poisson multinomial distribution. It addresses the limitation of prior approaches that assume i.i.d. instance-label pairs, showing that these methods underperform when this assumption is violated. Experiments on benchmark datasets with various bag construction strategies demonstrate that LLP-MM is both effective and computationally efficient.
By Tianhao Ma, Wei Wang, Yivan Zhang, Dong-Dong Wu, Takashi Ishida, Gang Niu, Masashi Sugiyama
The paper introduces “P…IBench”, a benchmark for proactive video understanding that tests daily perception, risk prediction, and intervention in real‑life streaming scenarios. It evaluates 12 video‑language models, finding that current systems struggle to make well‑calibrated response decisions. The authors also create new proactive training data and train “P…I-Assistant-4B”, which shows uneven improvements across the benchmark’s capabilities.
By Boyu Cui, Junhao Luo, Zhihao Wang, Linghan Chen, Zejun Li, Jiwen Zhang, Siyuan Wang, Zhongyu Wei
The paper introduces Smooth‑L1 regularized Echo State Networks (SL1ESN) that encourage sparsity in the readout layer while remaining differentiable for gradient‑based training. Using a twice‑differentiable Smooth‑L1 approximation, SL1ESN is trained centrally and via a second‑order federated aggregation scheme, achieving 75–95% sparsity across seven time‑series classification benchmarks. The resulting sparse models reduce storage, inference cost, and communication overhead, with only minor accuracy loss compared to dense baselines.
By Gengcheng Lyu, Manuel R\"oder, Frank-Michael Schleif
ReFlow is a system‑algorithm co‑designed framework for efficient reinforcement learning (RL) post‑training of large language models (LLMs). It introduces replay‑driven decoupling, allowing recent trajectories to be reused multiple times to amortize rollout costs while limiting policy staleness. The architecture features a dual replay cycle, an elastic resource pool, and an adaptive scheduler, achieving up to 2.06× higher training throughput compared to existing synchronous and asynchronous RL systems without sacrificing model performance.
By Yukun Tian, Junwei Su, Haofeng Xu, Lansong Diao, Zhengping Qian, Weibo Zhao, Wenchao Zhou, Ruiting Zhou, Chuan Wu
The paper presents a physics‑aware distillation of a large Grid Foundation Model (GridFM) for real‑time edge inference. A 20.1 M‑parameter, 12‑layer teacher is compressed into a 0.62 M‑parameter, 6‑layer student, achieving a 32.5× reduction in parameters and halving message‑passing depth while improving accuracy on active‑power dispatch by up to 48 % and always outperforming the teacher on reactive‑power dispatch. The distillation objective combines masked power‑flow loss with output, physics‑residual, latent‑state, and attention transfer, though the physics‑residual term proved detrimental in this setting.
By Vlad Alexandru S\u{a}nduleac, Shahariya Rasheed Rayamarakkarveettil Puthanpurayil, Noman Khan, Remus Teodorescu
The paper presents a neural evaluator that predicts Lean 4 theorem‑proving verdicts and compiler feedback, trained on 4.84 million labelled programs. A two‑stage approach—first screening candidates with a validity classifier, then scoring positives with a feedback model—achieves 87.8 % accuracy on a 68,932‑program benchmark and processes 3.5 candidates per second on an H100 GPU, outperforming a 64‑worker Lean baseline. Integrating this evaluator into a hybrid reward system reduces GRPO training steps by about 31 % while maintaining comparable miniF2F pass rates.
By Lazar Milikic, Etienne Bamas, Emmanuel Abbe, Viktor Kun\v{c}ak
ChronoSheaf introduces a joint spatio-temporal spectral filtering framework for cellular sheaves, extending sheaf neural networks beyond instantaneous spatial operators. It defines a spectral sheaf Laplacian that simultaneously governs signal propagation across both graph structure and time, and employs a Dual Spectral Filter to broaden receptive fields without enlarging the sheaf. The model incorporates Transitional Restriction Maps to allow asymmetric temporal influence and demonstrates superior performance on various spatio-temporal forecasting benchmarks.
By Abeer Mostafa, Zahra Ahmadi
The paper investigates whether a frozen transformer can use another model’s KV cache as a substitute for its own, finding that this approach fails because the reader only accepts caches within a narrow tolerable set. Experimental results show that none of eight substitution methods improve performance over a baseline, while a different strategy—adding the sender’s cache as extra context and only adapting queries—does outperform the baseline. The authors conclude that pursuing faithful cache substitution is misguided and that augmenting the reader’s own cache is the more promising direction.
By Adam Deng, Shicheng Liu, Yada Zhu
MIDI-REPA is a framework that aligns symbolic music representations with intermediate layers of a flow-matching DiT model to improve text-to-music generation. By deriving MIDI supervision from automatic music transcription and aligning a MuScriptor-derived piano roll with an early DiT layer throughout the flow trajectory, the method achieves competitive performance on the ATTM objective metrics without extra training data or techniques. The approach is trainable on a single RTX 3090 in about 31 hours and is supported by released transcription pipeline, training code, and audio samples.
By Song-Ze Yu, Fang-Duo Tsai, Yi-Hsuan Yang