The paper introduces a method called Addition by Subtraction to strengthen safety alignment in large language models. By gradually removing identified safety‑critical neurons during finetuning and re‑optimizing preferences, the authors reduce the model’s reliance on sparse components that can be targeted by pruning or ablation attacks. Experiments on four models from three vendors show that this approach improves robustness to both white‑box and black‑box jailbreaks, lowers attack success rates by up to 90%, and enhances truthfulness while preserving general capabilities.
By Hamid Dashtbani, Alexandra Dmitrienko
LifeVal is a new benchmark that evaluates AI model performance on personalized agentic consumer tasks, covering over 1.7 trillion US dollars in annual willingness-to-pay across 148 task categories such as managing finances and finding childcare. The benchmark was built using a survey of more than 3,800 consumers and cross‑referenced with national time‑use studies. Task creators, who met strict expertise thresholds, designed AI evaluation tasks personalized with real ChatGPT conversations, memories, reference files, and personal context, and all tasks underwent at least three rounds of human quality control before inclusion.
By Jenny Yang, Elizabeth Proehl, Anjan Bharadwaj, Emily Fan, Charlotte Cole, James Kim, Noam Brown, Phoebe Thacker, Amelia Glaese, Tejal Patwardhan
The paper introduces a method for efficient agent evaluation by using cached traces from task executions. It proposes summarizing each trace with an LLM and embedding the summary to capture task-relevant signals while reducing system-identifying noise. Using these representations, the authors build low-dimensional system embeddings and predict a new system’s benchmark score, achieving lower error than baseline models on SWE-bench Verified and Terminal-Bench 2.0 with only a single executed task.
By Ben Johnson, Hayden Helm
FrontierPhysics is a new end‑to‑end benchmark for AI agents in physics, comprising 56 tasks created by 42 physicists across nine domains of theoretical, computational, and experimental physics. Each task requires an agent to review literature, devise a research plan, conduct experiments, and produce structured outputs and manuscripts, with evaluation based on deterministic verifiers and expert rubrics. In tests of 22 models, none exceeded 46 of 168 rollouts, revealing that agents struggle mainly with rigorous reasoning and expert intuition rather than physics knowledge or execution.
By Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma, Jiechao Feng, Bo Chen, Xiangkai Sun, Jiarui Liu, Jinen Guo, Zeyu Zhou, Xu Huang, Zhuohan Li, Haoyu Li, Heguang Lin, Lijie Ding, Yadong Zeng, Kangqi Zhang, Xinge Liu, Hanwen Liu, Zhenghong Li, Alexander von Recum, Yuchen Zhu, Felix Schwarzfischer, Yizhou Zhao, Jicheng Wang, Binxu Li, Qian Yu, Chan Li, Cheng Dai, Di Wang, Haoyu Zhang, Jiankai Sun, Kaituo Feng, Mian Qin, Ruihong Yin, Yiqun Luo, Weiwei Wu, Shihong Pan, Tsung Yeh Hsieh, Xiao Huang, Yan Zhang, Sihan Li, Zirui Ren, Zengji Tu, Kyoung Whan Choe, Ke Sun, Vincenzo Galia, Vaisakhi Mishra, Jiashu He, Hongyang Wang, Zhiheng Yang, Shubham Gaur, Yifan Gu, Shuyi Wang, Zihao Zuo, Steven Dillmann, Yiyou Sun, Xinyang Han, Hanchung Lee, Molei Tao, Yue Zhao, Shilong Liu, Yuchen Wu, Xuandong Zhao, Xiangyi Li, Florian Brand, Gerbrand Ceder, Hartmut Haeffner
BEAKER is a new benchmark for evaluating embodied actionability, knowledge, experimentation, and reasoning in self‑driving chemical laboratories. It contains 1,000 expert‑curated question‑answer pairs across 11 image and video task types, covering affordance grounding, trajectory planning, equipment and state recognition, safety, apparatus setup, operation, step reasoning, and temporal procedure reasoning. The benchmark is manually created and reviewed by chemistry and embodied AI experts, and it tests 28 different multimodal large language models in an offline, single‑turn setting.
By Fei Lin, Tengchao Zhang, Ziyang Gong, Xiaotong Yu, Bohan Zhang, Yifan Zhou, Qihao Yang, Ji Dai, Dong Li, Yining Jiang, Qinghua Ni, Jun Huang, Qiang Zhang, Yue Zhou, Zhihang Zhong, Yonglin Tian, Fei-Yue Wang, Xue Yang
The paper introduces a training‑free ‘socket’ that connects a DNA foundation model to a frozen large language model (LLM) using a single ridge‑regression plug. The plug maps Evo 2 DNA features to the LLM’s internal representation of a gene’s symbol, name, and description, replacing hidden states at placeholder positions without back‑propagation. Evaluated on the GeneHop benchmark, the socket consistently outperforms shuffled DNA across four hosts, achieving significant gains in accuracy and demonstrating robust gene‑identity transfer without additional training.
By Arash Fatehi
The paper introduces Omni-Encoder, a single SigLIP-2 Vision Transformer that jointly processes image, video, audio, and audio‑visual data using fully shared parameters. It replaces separate modality encoders and anchor‑based alignment with a unified architecture that tokenizes audio, applies multimodal rotary position embeddings, and trains all modalities end‑to‑end through alternating‑modality batches. Omni‑Encoder achieves competitive performance on 16 benchmarks across all four modalities, outperforming individually trained counterparts, and the authors plan to release the code and models publicly.
By Xiaofei Sun, Peng Wang, Huayao Liu, Xiaoxiao Xu, Jun Song, Deqing Li, Lin Qu
The paper introduces Context as Covariates (CoCo), a forecasting framework that combines Large Language Models (LLMs) and Time Series Foundation Models (TSFMs). It generates realistic multimodal datasets by annotating real-world time series with LLMs and aligns the two models through staged fine‑tuning: first using Group Relative Policy Optimization to improve the LLM’s covariate generation, then fine‑tuning the TSFM on those covariates. Experiments on synthetic and real benchmarks across finance, economics, weather, traffic, and security show that CoCo can outperform state‑of‑the‑art TSFMs and LLM forecasters by up to 12% MAE.
By Hemanth Ram, Sai Shankar Narasimhan, Shubhankar Agarwal, Sandeep Chinchali
Bio‑VQA is a new, large-scale visual question answering benchmark for biology, comprising 1.86 million questions over 217 k images from 11 datasets. It tests four key aspects of biological visual understanding—taxonomic reasoning, attribute grounding, spatial reasoning, and individual re‑identification—using a graded scoring system that rewards near‑misses more than distant ones. Evaluation of 18 open‑weight vision‑language models shows that general visual capabilities do not transfer well to biological tasks, with markedly lower accuracy on counting, taxonomy, trait measurement, and re‑identification compared to everyday object benchmarks.
By S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma
The paper introduces VideoMogaNet, a video recognition model that addresses computational inefficiencies in temporal modeling by using parallel multi-order processing with depthwise-only convolutions and implicit gating-based aggregation, achieving linear complexity in channel count. It also proposes a dual interaction between spatial and temporal dimensions to enhance spatio-temporal representation without added cost. VideoMogaNet matches or surpasses state‑of‑the‑art action recognition performance on benchmarks such as UCF101, HMDB51, Kinetics‑400, and Diving48 while significantly reducing temporal branch computation.
By Kevin D. Nguetche, Lilian Hollard, Eric Granger, Marco Pedersoli
The paper introduces SME-C, a human‑AI workflow that combines lightweight expert critiques with LLM‑driven refinement to create high‑quality, domain‑specific evaluation datasets and rubrics. In a financial business brief domain, SME‑C gathered 3,853 critiques from 411 professionals and used two LLM refinement strategies—one guided by written critiques and another by preference votes—to improve response quality. Blind validation with 100 additional professionals showed both strategies significantly outperformed the experts’ top‑ranked baseline, with critique‑based refinement yielding more domain‑grounded improvements and vote‑based refinement better at removing irrelevant content.
By Honson Ling, Kidus Admassu, Stephanie Rosenthal
DARE (Difficulty Analysis for Repair Enhancement) is a framework that measures, predicts, and interprets the difficulty of coding repair tasks using a continuous difficulty metric based on failure rates across multiple agents. It employs 151 static features to predict difficulty, analyzes contributions from seven feature groups to identify task-specific characteristics, and selects tailored agent support accordingly. Experiments on 2,594 tasks across four benchmark partitions show that DARE’s predictions correlate well with actual difficulty and that matching support to salient feature groups improves repair success.
By Jikwang Lee, Seungwook Jung, Jaechang Nam
PEARL is a neural contextual bandit algorithm that selectively forgets only the dimensions of user preferences that have drifted, by reversing the Gram matrix update along feature directions where predictions are surprisingly wrong. The method achieves lower cumulative regret that scales with the number of shifted users rather than the entire population, and outperforms greedy, sliding‑window, and discounted baselines on the PRISM personalized LLM response selection benchmark, improving scores by 2.5 to 4.0 points over the best neural bandit baseline.
By Rashmi Gangadharaiah, Balakrishnan Narayanaswamy
The study evaluates the interventional validity of clinical trial simulators by creating an intervention‑blind reference model that withholds intervention identifiers. Using 9,996 within‑trial contrasts from 7,207 phase 2/3 trials, the authors measure how much a simulator’s accuracy improves when it knows the intervention. Results show that a registry‑trained gradient‑boosting model performs poorly on interventional skill, while a large language model reading only the protocol achieves positive interventional skill, and perturbation tests confirm that the model relies on the intervention information.
By Alicem Koyun
SysPilot is a cross‑library performance autotuning system that uses Library Performance Contracts (LPCs) to describe each library’s resource demands and observability signals. By synthesizing contracts from documentation and code, composing them for an application, and validating conflict hypotheses through profiling, SysPilot selects configurations that mitigate resource contention among independently developed libraries. In benchmark tests, SysPilot matches the performance of manually tuned autotuners and outperforms general‑purpose autotuners by up to 38% while requiring three times fewer runs.
By Yineng Yan, Haowei Chen, Christopher J. Rossbach
The paper proposes a drift‑adaptive method for Domain‑Incremental Learning that tracks representation drift for each past domain‑class pair. By measuring how much a concept’s current representation deviates from a fixed anchor, the method assigns stronger preservation to high‑drift pairs and down‑weights highly drifted domains during prototype construction. Experiments on DomainNet, Office‑Home, CORe50, and CDDB‑Hard show that this approach maintains a strong accuracy‑forgetting balance while using less training memory and the smallest storage compared to other methods.
By Hyunmin Kong, Gyutae Oh, Jitae Shin
The paper introduces GEdit, a geometry-aware framework for editing large language models that leverages a Low-Rank Plus Diagonal factor model to capture the low-rank structure of Transformer FFN hidden representations. GEdit applies low‑rank regularization to protect core knowledge and adaptive regularization to adjust constraints based on the difficulty of each edit, allowing harder edits to succeed while preserving easier ones. Experiments on GPT2‑XL, GPT‑J, and LLaMA‑3‑8B with up to 10,000 continuous updates demonstrate that GEdit outperforms state‑of‑the‑art baselines in both editing success and knowledge preservation.
By Bingqing Liu, Wei Liu, Jun Wang, Xiaobo Sun, Yuhua Li, Ruixuan Li
Quo Vadis, Driving Scene Reconstruction? Occlusion-Aware Gaussian Densification introduces OccSplat, a feed-forward framework that addresses the limitation of single-observation 3D Gaussian Splatting methods in driving-scene reconstruction. OccSplat identifies occluded regions lacking Gaussian support via Gaussian Occlusion Mapping and then generates new Gaussian primitives for these areas through Gaussian Occlusion Completion. By integrating the predicted primitives into the original representation, OccSplat enhances novel-view synthesis without requiring per-scene optimization, achieving significant improvements on the nuScenes benchmark, especially in occluded regions.
By Dung Do Tien, Kien Nguyen Do Trung, Kiet Dang Vu, Trung Thai Tran, Duc Dung Nguyen
MonoLaDN introduces a language-guided denoising framework that transfers geometric reasoning from a frozen vision‑language model (VLM) into a monocular 3D object detector. During training, the VLM generates structured reasoning about occlusion, depth, dimension, and angle for each object, which is encoded into queries, perturbed with noise, and used to train the detector to reconstruct the clean reasoning. This process embeds the VLM’s geometric priors into the detector’s depth‑guided features, allowing the deployed model to remain purely vision‑based with no added latency or parameters, and achieves state‑of‑the‑art performance on KITTI 3D and nuScenes while generalizing well to new domains.
By Kiet Dang Vu, Cao Le Minh Khoa, Nguyen Viet Tri Pham, An Nguyen Gia, Tam Le Thuc Ho, Duc Dung Nguyen
The paper introduces CASE-Bench, a benchmark of 145 repository-level tasks derived from real CVEs across 51 projects, each run in Docker with functional tests and proof‑of‑concept exploits. It evaluates 13 context‑construction methods and six controls across six LLMs, showing that functionality‑oriented context improves functional pass rates by up to 8.2 pp but only 3.7 pp for security, that omitting safeguards can drop secure pass rates by 8.1 pp, and that coding agents see relevant safeguards in only 27‑36 % of trajectories. These results highlight how repository context shapes the security of LLM‑generated code and guide the design of security‑aware context construction and evaluation.
By Jiongchi Yu, Keke Lian, Junqing Wang, Zhikang Fan, Zhaoxuan Li, Yongheng Huang, Ziming Zhao, Dong Zhang