The paper investigates why certain synthetic relational datasets lead to better relational foundation models. By training four Relational Transformer checkpoints on data from four different generators, the authors trace a measurable property of the data—specifically the predictive necessity of cross‑table information—to the emergence of relational computation in the models. They find that the RelDiff generator yields the largest predictive gain from foreign‑key‑linked parents, and its model uniquely responds to foreign‑key interventions, a dependence that persists across random initializations and grows with corrupted links. Disrupting this mechanism during downstream inference eliminates RelDiff’s advantage on relational tasks while leaving structure‑insensitive models largely unchanged, thereby linking synthetic data properties to learned mechanisms and downstream behavior.
By Shivam Dubey, Mohamed Bouadi, Nassim Bouarour, Varun Kulkarni, Aditya Tanna, Vinay Kumar Sankarapu
The paper argues that current time‑series forecasting benchmarks are too narrow, rewarding low held‑out error while failing to capture real‑world failure modes such as structured input anomalies, regime shifts, and sensor faults. It proposes scenario‑grounded stress testing, where each test includes historical data, future targets, a semantic scenario, a failure operator, and a difficulty level, to make evaluation interpretable and deployment‑relevant.
By Yuyang Zhao, Lian Xu, Hao Xue
Physical AI Smart Spaces is the first benchmark that offers large‑scale, multi‑class, multi‑camera 3D perception data for indoor smart spaces. It includes over 280 hours of synchronized 1080p footage from nearly 1,800 cameras in warehouses, hospitals, and retail venues, with automatic annotations for identities, 2D and 3D bounding boxes, camera calibration, and depth. The benchmark spans synthetic generation, appearance augmentation, and real‑world Sim2Real evaluation, and introduces a 3D version of Higher Order Tracking Accuracy (HOTA) for evaluating multi‑class 3D box tracking.
By Yuxing Wang, Yizhou Wang, Anqi Li, Shuo Wang, Sameer Satish Pusegaonkar, Haoquan Liang, Jiajun Li, Shenxin Jiang, Jianhe Yuan, Shangru Li, Tongwei Dai, Zihao Chen, David C. Anastasiu, Sujit Biswas, Xunlei Wu, Zheng Tang
The paper introduces an Embodied Neurocomputation framework that tackles the challenge of encoding and decoding between silicon computers and biological neural networks (BNNs). Using a large‑scale parameter sweep, the authors optimized over 1,300 encoding configurations for a BNN agent navigating an odor‑style gradient in a simulated grid‑world, running more than 4,000 hours of real‑time interactions. Twelve configurations consistently outperformed silicon‑based DQN agents within the same interaction budget, demonstrating the framework’s potential for scalable, goal‑oriented learning with BNNs.
By Johnson Zhou, Daniel Tanneberg, Forough Habibollahi, Alon Loeffler, Kiaran Lawson, Valentina Baccetti, Kwaku Dad Abu-Bonsrah, Candice Desouza, Finn Doensen, Bradley Watmuff, Daria Kornienko, Azin Azadi, Justin Leigh Bourke, Bernhard Sendhoff, Brett J. Kagan
MLCommons Jailbreak Benchmark v1.0 is a new methodology for assessing how well large language models resist single‑turn, text‑based jailbreak attacks. It evaluates eight open‑weight systems with 264 seed prompts across eleven hazard categories, using the AILuminate Assessment Standard v1.4 to measure the Resilience Gap—the difference in safety performance between baseline and adversarial conditions. The benchmark found that unsafe‑response rates rose from 11.08% to 18.65% under jailbreak conditions, yielding an average Resilience Gap of 7.57% and highlighting variability in attack effectiveness and evaluator reliability.
By Carsten Maple (Victor), Cagatay Yucel (Victor), Isaac Holeman (Victor), Chris Knotz (Victor), Peter Mattson (Victor), James Goel (Victor), Jonathan Petit (Victor), Sean McGregor (Victor), James Ezick (Victor), Abhishek Kumar (Victor), Alicia Parrish (Victor), Murali Emani (Victor), Kashyap Iyer (Victor), Faiza Khan Khattak (Victor), Washington Mbonu (Victor), Daniel Machlab (Victor), Eileen Long (Victor), Shaona Ghosh (Victor), Jibin Varghese (Victor), Roman Lutz (Victor), Andrew Gruen (Victor), Bennett Hillenbrand (Victor), Prabal Gupta (Victor), Mohammed Serrhini (Victor), Dhivya Nagasubramanian (Victor), Aakash Gupta (Victor), Jun (Victor), Lu, Kurt Bollacker, Chang Liu, Jonathan Petit, Cong Chen, Jean-Philippe Monteuuis, Brent Miller, Apurv Verma, Roman Eng, Armstrong Foundjem, Mohammed Serrhini
WAM-OPD introduces a method for post‑training improvement of World Action Models (WAMs) by collecting student rollout histories and querying a stronger teacher for paired video and action targets. The student learns from both modalities while preserving its one‑step generation capability at deployment. Experiments on 12 RoboTwin 2.0 tasks and four real‑robot tasks show success rates rising from 33.8% to 65.7% and from 51.4% to 64.6%, respectively, with joint video‑action supervision outperforming single‑modality approaches.
By Liuhaichen Yang, Zhengyang Zhong, Hanshang Zhu, Ningwei Bai, Qichen Yin, Zhi Han, Jiarui Qin, Zhuang Jiang, Chenchao Sheng, Hanbo Ma, Junkai Liu, Junkai Sun, Dongcheng Lyu, Bo Liu, Yi Dong, Zezhi Tang
VTBench is a new benchmark that evaluates visual tokenizers (VTs) used in autoregressive image generation. It assesses VTs on image reconstruction, detail preservation, and text preservation across diverse scenarios, revealing that continuous VAEs outperform discrete VTs in maintaining spatial structure and semantic detail. The study also explores GPT‑4o’s potential autoregressive behavior and releases the benchmark publicly to encourage development of robust, open‑source VTs.
By Huawei Lin, Tony Geng, Zhaozhuo Xu, Weijie Zhao
UniDynamics is a diffusion-based framework that generates future 4D dynamic scenes—comprising RGB, depth, and optical flow—from a single event-RGB pair, without needing long histories or control priors. It introduces an Event Latent Enhancement module to align event data into conditioning features and a Perceptual Dynamics Space within a multi-scale U‑Net to decouple and adaptively interact depth and flow, enforcing geometric and motion constraints for coherent predictions. Experiments on VKitti2 and DSEC show state‑of‑the‑art performance, producing high‑quality, temporally coherent, and 4D‑consistent predictions even under high‑speed motion blur.
By Daikun Liu, Xin Zhan, Teng Wang, Xiaoping Wang, Changyin Sun
Cephalonauts One is a 3 Tesla fMRI dataset featuring 30 hours of brain activity recorded from three healthy subjects while they listened to native-language audio podcasts. The release includes the raw fMRI data, corresponding podcast audio, transcript annotations, and derived stimulus embeddings, making it the deepest naturalistic speech fMRI dataset available. A brain‑decoding benchmark is introduced, framing audio segment retrieval as a task where decoders must match fMRI activity to the correct time‑aligned podcast segment, with standardized splits, metrics, and baseline models provided.
By Antoine Collas, Louis Jalouzot, G\'eraud Ilinca, Corentin Caris, Romain Valabr\`egue, Ahmed Hassayoune, David Goncalves, Madeleine Hueber, Thadd\'ee Delebarre, Julien Savatovsky, Clara Fonteneau, Charles Maussion, Bertrand Thirion, Alexis Thual
The paper investigates whether a proximal policy optimisation (PPO) agent can learn to act as a broker in a continuous‑time broker‑trader game that has an analytically solved optimal strategy. By discretising the continuous‑time reward and validating the implementation, the authors show that a PPO agent with a feed‑forward neural network can approximate the reference action when there is no uninformed order flow, but struggles under stochastic uninformed flow, with critics failing to rank actions reliably. The study further demonstrates that a causal certainty‑equivalent controller based on observable history outperforms PPO under partial information, and that freezing the analytical policy and fine‑tuning PPO after a change in execution cost yields a measurable improvement.
whyItMatters:"The analytical solution serves both as a diagnostic benchmark for RL performance and as a practical starting policy that can be adapted to changing market conditions, illustrating how theoretical finance models can guide and improve reinforcement learning in complex financial control tasks."
By Siu Tung Wong (Institute of Finance and Technology, University College London), Carlo Campajola (Institute of Finance and Technology, University College London, UZH Blockchain Center)
LoGo introduces a hybrid reward system combining global and spatially localized rewards to improve 3D consistency in camera-controlled video generation. The local reward offers fine-grained credit assignment, reducing object shifts and artifacts, while the global reward maintains camera following and overall video quality. Evaluations on DL3DV and the new TrajectoryBench benchmark demonstrate LoGo’s advantage over existing post-training techniques.
By Ziqi Ma, Shreya Sharma, Mohamed El Banani, Katja Schwarz, Chongjie Ye, Chao-Yuan Wu, Li Fei-Fei, Ben Mildenhall, Georgia Gkioxari, Justin Johnson, Gowthami Somepalli
FASTDIAR is a frame‑level speaker encoder designed for real‑time streaming diarization on a single CPU thread. It converts a state‑of‑the‑art speaker recognition architecture into a causal model that emits an embedding every 80 ms from a two‑second window, and pairs it with online clustering that updates only when the stream’s self‑similarity changes. Trained via distillation from an utterance‑level teacher, it achieves the highest accuracy on low‑overlap benchmarks, degrades less with more speakers, and runs five times faster than real time.
By Nikita Torgashov, Okan K\"op\"ukl\"u
HyperLogic is a forward‑construction benchmark for Chinese logical reasoning that decouples problem authoring from answer generation. It employs a multi‑agent workflow to harden undergraduate‑authored seeds, translate items into executable finite‑domain models, and adjudicate solver‑derived answers under human oversight. The benchmark includes a Base set of 195 items (922 sub‑questions) and a Hard set of 100 items, revealing significant performance gaps among frontier models and demonstrating the challenges of faithful formalization even with tool support.
By Ming Zhang, Qiyuan Peng, Yinxi Wei, Yujiong Shen, Kexin Tan, Yuhui Wang, Zhenghao Xiang, Junjie Ye, Zhangyue Yin, Zhiheng Xi, Shihan Dou, Weikang Wang, Yuhao Zhang, Tao Gui, Ruizhi Yang, Qi Zhang, Xuanjing Huang, Alex Chen, Maxm Pan
Memristor-based analog compute-in-memory architectures promise efficient deployment of Large Language Models, yet their intrinsic non-idealities degrade reasoning performance across benchmarks. The study evaluates how these non-idealities affect LLM reasoning and tests three training-free mitigation strategies: thinking mode, in-context learning, and module redundancy. Findings show shallow layer redundancy boosts robustness, thinking mode helps only at low noise, and in-context learning shortens output with a modest performance cost.
By Taiqiang Wu, Yuxin Cheng, Chenchen Ding, Runming Yang, Xincheng Feng, Wenyong Zhou, Zhengwu Liu, Ngai Wong
ForestQuery is a new framework for unified forest point cloud segmentation that incorporates boundary-aware and spatially anchored query learning. It explicitly models boundary uncertainty to improve instance query construction and uses learnable 3D anchors to encode forest vertical stratification for semantic queries. Experiments on public benchmarks and a real‑world dataset show consistent gains in both individual‑tree and semantic segmentation.
By Zhihao Zhan, Le Tao, Yifei Tian, Xin Liu, Jie Yuan
PLCWorld is a closed‑loop execution environment and benchmark that evaluates large language model (LLM)–generated programmable logic controller (PLC) programs by coupling Structured Text (ST) execution with simulated plant responses and sensor feedback. It includes 100 synthetic tasks and 473 task‑condition pairs across Motion Control and Material Handling, with difficulty levels based on control‑dependency scope. The benchmark reports Task Success and Safety Violation separately and validates results through practitioner review, reference programs, counterexamples, and comparisons with independent ST runtimes, revealing performance differences among six LLMs and four generation‑and‑verification workflows.
By Yunji Kim, Yunseok Lee, Hyunwoo Seo, Jaerim Choi, Woojin Lee
The paper introduces Success-Referenced On-Policy Distillation (SR-OPD), a method that reduces teacher computation in on-policy distillation by selectively providing teacher supervision only for prompts where the student has both successful and failed rollouts. SR-OPD uses successful rollouts as references to prioritize failed rollouts that diverge significantly from the successful ones, while also considering estimated teacher-input cost. Experiments on three teacher-student pairs across six mathematical reasoning benchmarks show that SR-OPD requires only 3.46‑5.02% of the teacher-input tokens of Vanilla OPD in a one-pass setting, yet maintains comparable reasoning performance, and further validates its design choices under a 5% teacher-input budget.
By Xiang Chen, Futao Su, Kong Wang, Jiayi Chen, TanLin Li
TRACE is a reproducible benchmark for electricity price forecasting that pairs 7,300 zone–day instances from five U.S. market zones with official operational text available at the forecast cutoff, ensuring no post‑cutoff information leakage. The benchmark evaluates semantic alignment and forecasting value, showing that the text aligns with central movement and tail risks in ground‑truth prices, especially for upper‑tail risk. Forecasting models using TRACE’s text achieve a median 7.4% reduction in upper‑tail pinball loss, while a controlled ablation that mismatches text reverses these gains.
By Xinyi Yi, Moy Yuan, Ioannis Lestas
The paper investigates the geometric capacity of vector retrieval systems, focusing on the maximum recall achievable with a fixed document index. It demonstrates that, on real-world benchmarks, single-vector query encoders often underperform relative to the index’s potential. The authors provide theoretical evidence that learning such encoders can be computationally hard, constructing a task where a simple neural network can achieve perfect recall while any statistical-query learner would need exponentially many queries to surpass random chance.
By Anders Wikum, Nina Mishra, Amin Saberi, Tal Wagner
The paper introduces Calibrated User Embeddings (CUE), a framework that encodes real user sessions into continuous representations and decodes them into persona commands to steer large language models (LLMs) as user simulators without additional training. CUE enables user-conditioned replay of past interactions and generates novel personas that better align with real-user success rates and failure patterns. Evaluations on the τ²‑Bench dataset show that CUE‑driven simulators commit fewer simulator‑attributed errors, more accurately reproduce real‑user failure modes, and maintain competitive user fidelity across various tasks and LLMs.
By Anjali Kantharuban, Jonas Mueller