The paper examines how profit motives influence safety investment in AI, focusing on a supply chain with an upstream large language model (LLM) provider and multiple downstream firms. Using a game‑theoretic model, it shows that downstream competition tends to drive firms to invest more in safety than would maximize industry profit, while the upstream provider may underinvest because it does not capture all the benefits of safer models. The study also explores how regulatory interventions could shift incentives and affect overall model safety.
By Nikita Tsoy
TASTE (Task-Aware STEin operators) is a framework that links distribution shift to a model’s input sensitivity using Stein operators. It provides a geometric interpretation of shift as a projection onto the model’s sensitivity field, offers coordinate‑wise localisation, and delivers interpretable per‑pixel diagnostics for image data. Experiments on Gaussian shifts, MNIST, and CIFAR‑10/ImageNet‑1K benchmarks show that TASTE closely tracks task degradation and performs competitively with existing baselines, with gains varying by shift regime.
By Micha{\l} Kozyra, Gesine Reinert
QuasiLeak-Bench is a new benchmark that evaluates large language model (LLM) based privacy detectors on their ability to recover quasi‑evidence groups—sets of weak quasi‑identifiers that jointly reveal private attributes—while respecting sender‑specified disclosure policies. The benchmark uses a reverse‑construction pipeline to generate documents containing verified quasi‑evidence groups and applies automated quality control to produce reliable ground truth at scale. Experiments show that current privacy detectors perform poorly on this task, but a fine‑tuned 4B LLM can outperform the best prompted models, indicating the task is learnable.
By Yufan Zhu, Yangfan Jiang, Yang Fei, Shizheng Hou, Shubhankar Mohapatra, Xiaokui Xiao
The paper introduces multi-target few-shot prompting using diffusion language models, enabling the infilling of multiple masked regions within a shared canvas guided by examples. It addresses challenges in few-shot prompting by selectively masking the canvas to derive a likelihood-ratio reward that favors example-supported infills. Experiments on prompting benchmarks and software refactoring case studies show that demonstrations improve infilling accuracy and provide additional gains in several settings.
By Anton Xue, Alan Li, Rahul Saha
ZoomCap is a method for fine‑grained captioning that treats missing visual evidence as recoverable information. It uses a draft caption to guide conditional knowledge retrieval, OCR‑aware reading, coverage‑gap recovery, and adaptive visual zooming, focusing deeper crops on complex regions and deriving spatial relations from grounded geometry. The accompanying FactCycle framework evaluates caption claims against the source image and a reconstruction, distinguishing fabrication from generator loss, and the approach shows strong performance gains in both general and medical vision‑language pre‑training.
By Hao Shi, Jiafeng Wang, Yuzhuo Fu, Tao Zhang, Jian Bai, Juelu Zhang, Qing Cui, Chao Huang
MoCo is a Python library that consolidates 26 model collaboration methods for language models, enabling execution, benchmarking, and comparison at scale. It includes 25 evaluation datasets across reasoning, QA, code, safety, and more, allowing users to add their own data. Experiments show that most collaboration strategies outperform non‑collaborative models in 61.0% of settings, with top methods achieving up to 25.8% improvement.
By Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang, Zhaoxuan Tan, Jiajie Yan, Zhenyu Lei, Wenxuan Ding, Weijia Shi, Haojin Wang, Zhenting Qi, Yuru Jiang, Heng Wang, Chengsong Huang, Yu Fei, Jihan Yao, Yilun Du, Luke Zettlemoyer, Yejin Choi, Yulia Tsvetkov
The paper introduces a training‑free, target‑aware refinement method for 3D Gaussian language maps distilled from vision‑language features. By blending the visual embedding within user‑specified target regions with the corresponding text embedding using a convex combination, the method improves target localization while preserving open‑vocabulary queryability. The authors derive the optimal blending coefficient analytically, avoid hyperparameters, and demonstrate consistent gains across three benchmarks, including a real‑world robot navigation demo.
By Sibaek Lee, Kyeongsu Kang, Seongbo Ha, Yewon Jeon, Joonyeol Choi, Daegeol Ko, Jaehyuk Hur, Taemin Choi, Minsu Kim, Hyungjoo Ahn, Wonsok Yoo, Hyeonwoo Yu
The paper introduces SynOE-OD, a method for detecting out-of-distribution (OOD) objects in street scenes by generating synthetic outliers through text-guided inpainting and an open-vocabulary detector. These synthetic outliers are used to fine-tune a chosen object detector with an added OOD class, eliminating the need for auxiliary branches during inference. On the OoDIS benchmark, SynOE-OD achieves state‑of‑the‑art performance, outperforming the strongest open‑vocabulary baseline across multiple datasets while preserving competitive in‑distribution detection.
By Sadia Ilyas, Annika M\"utze, Klaus Friedrichs, Thomas Kurbiel, Matthias Rottmann
FlatClip is a method that leverages frozen image foundation models to predict fMRI activity by mapping cortical signals onto anatomy‑linked flatmaps and extracting features with a SigLIP2 encoder. Only a lightweight downstream probe is trained, avoiding fMRI‑specific encoder pretraining or fine‑tuning. Experiments on four resting‑state benchmarks show FlatClip surpasses ROI‑level baselines on HCP and ADNI, while its performance varies on PPMI and remains below the best voxel‑level models; on NSD visual‑fMRI decoding, restricting inputs to task‑active cortex improves recognition, and spatial perturbations confirm the importance of cortical arrangement and input‑region selection.
By Mo Wang, Wenhao Ye, Zihan Ning, Jiayu Zuo, Junfeng Xia, Hongkai Wen, Quanying Liu
The paper investigates when supervised learning can recover the true latent variables that generate data, focusing on two causal settings: generative (labels cause latents) and discriminative (latents cause labels). It shows that if the representation space has at least as many dimensions as the latent space, optimal encoders are linear; otherwise, identifiability fails and encoders become nonlinear. Additional constraints—such as non‑negative activations in the generative case or sparsity and spark conditions in the discriminative case—enable recovery of individual latent variables, linking nonlinear ICA, compressed sensing, and neural network interpretability.
By Junze Zhou, David Klindt
The paper investigates whether intrinsic tokenizer features can predict the performance of downstream language models. By training about 200 models across nine languages and evaluating numerous tokenizer metrics, the authors find that none of the existing metrics consistently forecasts model performance across different families, tasks, and languages. They propose a new linguistically grounded metric that captures aspects of tokenizer quality missed by prior measures, though its predictive power may still vary.
By Antonio Lopardo, Luis Frentzen Salim, Shikhar Srivastava, Kevin Jiang, Mridul Sharma, Catherine Arnett
The paper introduces On-Policy Reminder Distillation (OPRD), a method that enhances long-horizon coding agents by combining multi-scope reminder supervision with outcome-based routing. OPRD uses successful rollouts as autoregressive self-imitation targets and failed rollouts for reminder-conditioned reverse-KL supervision from a teacher that re-evaluates student prefixes. The approach improves task resolution rates by 11.3 points and cuts error events by 50.2% on several SWE-bench benchmarks.
By Yuechen Luo, Zhiyi Lai, Miao Ziqi, Zihang Xu, Jingxuan Xu, Songwei Yu, Wenqiang Zhu, Weihao Li, Junhao Fan, Haoyang Huang, Yanan Wu, Ken Deng
WEAVE is a framework that converts verified pull‑request histories into executable, dependency‑structured tasks for code agents. It organizes related PRs into bounded directed acyclic graphs (DAGs) using repository‑calibrated typed change loci, creating unified tasks that reflect real‑world code evolution. The framework also introduces Graph Credit, a deterministic, test‑grounded metric that measures partial progress consistent with dependencies, providing dense, verifiable rewards for reinforcement learning.
By Weihao Li, Zihang Xu, Zhongqiu Wang, Songwei Yu, Wenqiang Zhu, Yuechen Luo, Jingxuan Xu, Yanan Wu, Haoyang Huang, Ken Deng
The paper examines whether transcripts of model solutions can reveal capabilities after benchmark scores saturate. By simulating saturation with historical data, the authors test protocols that use only correctly solved transcripts to predict performance on unsaturated items. Pairwise judging of transcripts outperforms other methods, reducing prediction error by 5.3×, though its effectiveness can vary across distribution shifts.
By Morio Hamada, Jehyeok Yeon, Matteo Migliarini, Justin Olive, Magda Dubois
AlphaDiana is a measurement system designed to evaluate agents in verifiable reasoning tasks by providing modular interfaces for datasets, agents, environments, and scorers. It maintains harness-native control loops while separately managing file reuse and memory retention, enabling both macro-level analyses of entire agents and micro-level interventions on components such as tools, skills, and memory. Experiments on open-source models and harnesses across multiple benchmarks reveal that harnesses and component interventions do not consistently improve reasoning accuracy, and that token usage metrics must be interpreted alongside correctness.
By Zhanke Zhou, Zongze Li, Weikai Huang, Xuan Li, Chentao Cao, Xiao Feng, Xiangyu Lu, Jinbo Hu, Yi Xie, Nico Pelleriti, Shiyang Liu, Menghan Lu, Max Zimmer, Brando Miranda, Jiangchao Yao, Bo Liu, Sanmi Koyejo, Sebastian Pokutta, Bo Han
IneqAgent is a new framework that combines large language model reasoning with symbolic computation to prove inequalities in Lean4. It uses lemma-level Monte Carlo Tree Search to coordinate lemma construction, goal reformulation, and backend selection, employing sum‑of‑squares, Bernstein, and successive difference‑substitution backends to generate exact certificates. In tests on four benchmarks, IneqAgent with DeepSeek‑V4‑Flash solved 175 of 186 problems (94.09%) and produced Lean‑verified strengthenings of Dittert’s conjecture and a certification of the Holens–Djokovic $(4,4)$ inequality.
By Gaolei He, Jinhui Li, Ruobing Zuo, Xia Zeng, Zhengfeng Yang
The paper examines the vulnerability of skill-policy co‑evolution in self‑improving agents, showing that policies become fragile when the skill bank changes after training. It introduces CODA, a deployment‑oriented evaluation protocol that simulates skill drift and measures worst‑case success, consistency under meaning‑preserving edits, and resistance to corrupted skills. Across multiple methods and benchmarks, CODA reveals that worst‑case performance can drop 5–13 points, erasing a large portion of the gains achieved during co‑evolution.
By Yujie Feng, Xiaobin Wang, Jinluan Yang, Xiaonan Shi, Pengjun Xie, Philip S. Yu, Xiaoming Wu, Jieping Ye
The paper introduces Replay-Selected Hindsight Distillation (RSHD), a method that transforms failed attempts in GRPO training into useful supervision. By having the policy write corrective notes for each failure and re-solving the problem with those notes, RSHD admits only those notes that lead to higher success rates than the original attempts. This approach enables learning from groups where all rollouts fail and improves performance across multiple model families and domains.
By Sheldon Yu, Tong Yu, Nikki Lijing Kuang, Zhendong Chu, Zhenwei Tang, Sungchul Kim, Lina Yao, Julian McAuley, Junda Wu
Sciπvot is a 248‑case benchmark that evaluates how well large language model (LLM) agents perform scientific judgment during autonomous research. The benchmark is built from six months of real‑world experiments where senior researchers ran LLM agents on 26 projects across six scientific fields and repeatedly corrected the agents’ decisions. In each case, the agent must choose between its original plan and a human‑corrected alternative using only the project record available before the correction, allowing a direct comparison of LLM and human performance under varying reasoning effort and record access.
By Youran Sun, Xingyu Ren, Chugang Yi, Kejia Zhang, Di Wu, Yingkang Cao, Allen Zhou, Jianping Ye, Suying Liu, Jiaxuan Guo, Jianda Du, Weicheng Ye, Kecheng Liu, Hefan Zhang, Haixin Wang, Jie Xiao, Eric Yang, Haizhao Yang
CleanRM is a reward‑modeling approach that mitigates the bias caused by noisy preference data in RLHF. It calibrates sample‑wise noise ratios using a small set of clean feedback via optimal transport, aligning the joint distribution of noisy and clean preferences. The method then trains a reward model with a noise‑corrected surrogate loss, proven to be unbiased with respect to the clean risk, and outperforms existing denoising baselines in experiments.
By Licheng Pan, Haocheng Yang, Yuzhou Chen, Yichen Sun, Yuan Lu, Yunfei Teng, Jiajun Fan, Guangyi Liu, Zhixuan Chu, Hao Wang