The paper introduces noise‑rejuvenated Sequential Monte Carlo (NR‑SMC), a test‑time scaling method for text‑to‑image flow matching models. NR‑SMC perturbs resampled particles by altering their source noise and creating new intermediate states, allowing exploration without converting the sampler to an SDE. Experiments on Stable Diffusion 3, FLUX.1‑dev, and Z‑Image show consistent reward‑guided generation improvements, with ablation studies confirming the benefits of noise rejuvenation and look‑ahead refinement.
By Wenyi Lian, Zheng Zhao, Thomas B. Sch\"on
The paper investigates flow matching on manifolds, focusing on the 1D flat torus. It identifies a failure mode caused by a property called compatibility, where a likelihood-based flow matching objective can be globally optimized by a velocity field that actually pushes samples away from the data distribution. The authors propose a new compatible objective built from an exponential family over endpoints, which restricts the velocity field to feasible values on compact manifolds and improves performance on high-dimensional product tori, geospatial benchmarks on the sphere, and a large human pose dataset.
By Lucas Ng, Georgios Batzolis, Mark Girolami
The paper presents a technique that augments data‑driven polynomial chaos expansion surrogate models with uncertainty quantification through jackknife‑based conformal prediction. By leveraging the linearity of the regression model, leave‑one‑out residuals and predictions are computed analytically, avoiding repeated retraining and eliminating the need for a hold‑out dataset. The method is validated on benchmark models and applied to electrical engineering design cases, demonstrating accurate predictive intervals even with small, low‑accuracy training sets, while showing that more training data improves coverage and interval tightness.
By Dimitrios Loukrezis, Dimitris G. Giovanis
BARC-Bench is a new benchmark comprising 110 tasks across nine simulated workspace environments designed to expose the safety risks of underspecified user instructions for agents. The benchmark evaluates whether agents respect safety boundaries defined by workspace records, implicit environmental context, and unstated social norms, such as avoiding premature disclosure of an employee’s resignation. Results show that even the best model configurations fail to consistently act safely, with safe completion rates ranging from 9.1% to 72.7% and a 27.3% safety violation rate in the strongest setup.
By Kangkang Li, Kelvin Luu, Lily Ma, Tu Trinh, Ying Liu, Mohamed Elfeki
$M^3$ is an automated red‑teaming framework that models multi‑turn jailbreaking as a finite‑horizon Markov Decision Process. It lets an attacker LLM interact with a target LLM while using a state LLM to maintain a compact internal state. Across two benchmarks and six target models, $M^3$ outperforms existing methods, boosting attack success rates by up to 53% at five turns and improving consistency and risk escalation metrics.
By Yuqing Huang, Jack Ji, Jithin Pradeep, Sagarika Suresh, Jekaterina Novikova
PersonaScienceBench is a new benchmark comprising 200 tasks drawn from five existing benchmarks across four scientific domains. It introduces simulated researchers with distinct persona profiles who provide context and feedback during interactions, while the task objectives and grading remain unchanged. The benchmark evaluates scientific agents on how they collaborate with diverse researchers, assessing task performance, fact recovery, and self-repetition across eighteen model configurations.
By Siyang Wu, Weizhi Zhang, Chen Yang, Esther Xu Fei, Jorin Overwiening, Xu Pan, Jing Tang, Yixuan He, Kelvin C. Mo, Peiwen Li, Defu Cao, Xu Hu, Haixin Wang, Taoran Li, Haoming Li, Zheyuan Deng, Jingwen Zhang, Pengsong Zhang, Haichao Zhang, Guangze Luo, Yi Daniel Lu, Zihuai He, Chenxi Qiu, Dianzhuo Wang, Zhejian Peng, Shushu Liang, Wenyi Lu, Yifan Wang, Jintao Huang, Yijun Wang, Xiaomin Li, Yuexing Hao
arXiv:2610. 32177v1 Announce Type: new Abstract: Humans reason about the physical world by flexibly combining different forms of inference as a problem unfolds.
By Khai Loong Aw, Klemen Kotar, Baihan Zhang, Yifan Liu, Lilian Chen, Atlas Kazemian, Rahul Venkatesh, Daniel Yamins
The paper introduces TraceTrial-Bench, a benchmark that evaluates the reliability of AI agent harnesses across seven components—context, execution, governance, lifecycle, observability, tools, and verification—using 95 expert-level tasks with executable or state-based ground truth. It employs a Trajectory Tracing module to record agent execution, a two-layer evaluation framework to separately assess agent performance and harness reliability, and a Multi-source Evidence Acquisition module to gather evidence for LLM-based evaluation. The benchmark’s analysis of 3,420 runs across four harnesses and nine models shows harness scores between 59.8% and 72.0%, with 31.1% of runs succeeding despite failing at least one harness criterion, highlighting the need for auditable and reproducible harness evaluation.
By Zhiyong Cao, Hengjie Zhu, Yongxin Wang, Xuchen Li, Jintao Li, Shuxia Liu, Dunqiang Liu, Bo Wang, Kairan Zhang, Cong Lin, JiangDi Yu, Siyuan Liu, Sijia Yao, Yi-Liang Wei, Lingxu Ma, Aijia Wang, Anxin Chen, Zixian Wei, Xiang Ruan, Ruizhi Xu, Ruqian Shi, Baihua Xiao, Sicheng Zhou
Solvent is an 8‑billion‑parameter large language model designed for long‑horizon reasoning in multi‑step retrosynthesis. It learns to interleave synthesis planning, external availability checks, and adaptive route revision through a two‑round post‑training curriculum. Experiments show a 2.1× performance boost from reinforcement learning, achieving 61.86% accuracy across four benchmarks and outperforming mid‑scale language models and prior search‑based systems.
By Yifei Wang, Botian Wang, YiFei Wang, Jiangtao Feng, Xinbo Zhang, Yawen Ouyang, Yixin Wu, Lihao Wang, Xiaoqing Zheng, Xuanjing Huang, Hao Zhou
Async-RBench is a new benchmark that tests how large language model agents handle asynchronous task execution. It includes 200 executable tasks across eight scenarios, with a scenario controller that delivers results based on task-state conditions and an Event Response Contract that defines required changes, preservation, avoidance, and re-verification. The benchmark introduces the Dynamic Replanning Score (DRS) to evaluate both the response process and outcome, and finds that agents generally perform worse with asynchronous delivery compared to batched delivery, highlighting challenges in reliable asynchronous task handling.
By Detao Zeng, Guilin Ou, Nuo Chen, Yunhao Feng, Hongliang Wang, Qinli Yang, Junming Shao, Zhong Zhang
The paper introduces a framework for evaluating large‑language‑model agents by focusing on the interaction between the model and its execution harness. It defines five complementary metrics—Valid Completion Rate, Model‑Harness Fit, Cross‑Model Fit, Interaction Residuals, and Token Load—to assess how reliably agents finish tasks, how well harnesses support specific models, and the overall suitability across models. Using these metrics, the authors evaluate four general‑purpose harnesses and two native systems on six benchmarks, finding that harness rankings differ by model and task and that some harnesses consistently achieve high valid completion rates.
By Shao Zhang, Hangfan Zhang, Yuanyuan Shi, Hao Li, Min-Le Su, Kangcong Li, Jingping Wang, Chunjiang Mu, Chen Zhang, Yang Chen, Bo Zhang, Shuyue Hu
The paper introduces CASE, a Confusion-Aware Skill Evolution framework designed to improve Multimodal Emotion Recognition (MER) by addressing systematic confusion between semantically adjacent emotions. CASE builds a sparse, self‑evolving library of pair‑specific EmotionSkills that externalize affective expertise from frozen multimodal large language model (MLLM) backbones, allowing continuous refinement and transfer across different models. Experiments on CA‑MER, DFEW, and EMER benchmarks show that CASE significantly boosts accuracy, notably raising Qwen3‑VL‑8B’s performance from 46.80% to 59.13%.
By Siyu Wu, Zeyu Liang, Linghao Wang, Qiyun Zheng, Yi Yao, Ling Lo, Hongxia Xie
IGG is a new benchmark for Interactive GUI Grounding that requires agents to first interact with a GUI to make a target visible and then click it. It includes 601 tasks across 16 controllable environments and seven visibility-constraint sub‑types, with a compact action space of scrolling, moving, dragging, waiting, and clicking. Human participants achieve 94.6% accuracy, while GPT‑5.5 and the best open‑source VLM reach 79.4% and 31.6% respectively, highlighting a gap mainly due to the difficulty of executing reveal interactions.
By Kyeongseon Kim, Jiyeon Son, Tae-Hyun Oh
The paper introduces PICLE-PFN, a foundation model for longitudinal causal effect estimation that handles dynamic treatment policies. It represents policies via in-context history-decision demonstrations and is pretrained on synthetic data from longitudinal structural causal models, enabling it to predict conditional average potential outcomes in a single forward pass without task-specific training. Experiments on synthetic and semi-synthetic benchmarks show competitive RMSE and better uncertainty calibration while reducing deployment runtime compared to conventional estimators.
By Wenxin Chen, Kyra Gan, Mingyu Lu, Zilin Jing, Fei Wang, Hoifung Poon, Tristan Naumann, Peniel Argaw
Astro2Method is a benchmark comprising 500 expert-reviewed binary questions drawn from 346 astronomy papers, designed to evaluate large language models (LLMs) on selecting machine learning methods for astronomical problems. The study tests five LLMs, finding high accuracy (87.50‑92.55%) on Easy questions but lower performance (58.87‑64.52%) on Hard ones. Various retrieval-augmented and agent-based approaches, including web search and masked pipeline descriptions, show limited or inconsistent improvements, indicating that LLMs still struggle to infer suitable ML methods from related studies and domain knowledge.
By Xingzhong Fan, Hangfeng He, Fatemeh Nargesian
RubricCraft is a method that generates natural‑language evaluation rubrics for open‑ended, artifact‑producing language tasks by analyzing differences among independently sampled solver trajectories. It constructs rubric criteria based on how each attempt satisfies or violates task requirements, then applies and refines these criteria to better detect errors and recognize partial correctness without needing a single reference solution. On the SpreadsheetBench v2 benchmark, RubricCraft’s rubrics reduce the proportion of incorrect outputs that pass (29.5% vs. 42.6%) and improve ranking of outputs while maintaining consistency with the benchmark’s own checker.
By Hosein Hasanbeig, Priyanshu Gupta, Ashish Tiwari, Vu Le, Sumit Gulwani
MBABench-2.0 is a new benchmark that tests AI agents on building financial spreadsheets from scratch, featuring 101 original construction tasks that require complex modeling decisions, formatting, and layout design. Unlike previous fill‑in‑the‑blank tests, this benchmark evaluates spreadsheets on 129 quality criteria across 12 categories, addressing readability, maintainability, and auditability. Across 20 agents and 8 models, the best performer achieved only a 16% pass rate, highlighting a significant gap in current AI spreadsheet‑building capabilities.
By Julian Poeltl, Thomson Yen, Patrick Shea, Shubham Anil Varma, Dhruv Rastogi, Zihao Chen, Mengshi Zhang, Hongseok Namkoong
The paper introduces LLMMAD, an autoregressive multimodal anomaly detection framework that leverages Large Language Models for microservice systems. It addresses two key limitations of existing methods—ignoring modality-specific traits and the identity-shortcut problem—by embedding multimodal data into a unified sequence for LLM processing while preserving modality-specific information through dedicated reconstruction heads, and by using a prediction-driven autoregressive objective to avoid copying inputs. Experiments on three real-world datasets show LLMMAD outperforms state-of-the-art approaches, improving F1-scores by up to 3.1%.
By Yixiang Tang, Fagui Liu, Guoxiang Zhong, Xi Yao, C. L. Philip Chen
The paper introduces a defense for Agentic AI systems that use the Model Context Protocol (MCP). It identifies a vulnerability where hidden instructions in steganographic images can bypass text scanners and trigger malicious tool calls. The proposed solution, decode-time provenance binding, uses HMAC-authenticated trust labels, a provenance ledger, and context-taint tracking to mark decoded content as untrusted and block harmful tool calls, achieving 0% attack success in tests.
By Eric Yocam, Varghese Vaidyan, Paris Kalathas, Patrick Tague, Bruce DeBruhl, Yong Wang
The paper redefines structured generation as an ambiguity‑aware inference problem, arguing that natural language often allows multiple valid structured outputs. It introduces a multi‑valid learning framework that preserves these alternatives during training and uses graph‑based reasoning to compare candidate structures based on semantic alignment and relational consistency. Experiments on benchmark datasets show that this approach improves accuracy, stability, and robustness compared to traditional single‑output methods.
By Ratun Rahman, Atit Pokharel