DiscoPhon is a multilingual benchmark designed to evaluate unsupervised phoneme discovery from discrete speech units. It includes 6 development and 6 test languages that cover a wide range of phonemic contrasts, and requires systems to generate discrete units mapped to a predefined phoneme inventory using only 10 hours of speech from an unseen language. The benchmark assesses unit quality, recognition, and segmentation, and provides four pretrained multilingual HuBERT and SpidR baselines that demonstrate current models can produce units that correlate well with phonemes, though performance varies across languages.
By Maxime Poli, Manel Khentout, Angelo Ortiz Tandazo, Ewan Dunbar, Emmanuel Chemla, Emmanuel Dupoux
RGBD20K is a new large-scale RGB‑D semantic segmentation dataset featuring 20,000 image pairs and 160 fine‑grained categories, surpassing existing benchmarks like NYUv2 and SUN RGB‑D in both scale and semantic diversity. The dataset provides high‑fidelity annotations obtained through rigorous re‑evaluation and correction of prior labels, ensuring a clean ground‑truth foundation. Additionally, the authors introduce a score‑purified fusion (SPF) method that achieves state‑of‑the‑art performance across evaluated benchmarks, demonstrating the value of high‑quality multimodal information.
By Shaohua Dong, Zexuan Meng, Haiyan Sun, Bing Fan, Cuicui Zhang, Dylan Joseph, Kewei Sha, Yunhe Feng, Heng Fan
The paper argues that current embodied vision‑language planning benchmarks favor linguistic next‑token prediction over physically grounded next‑state reasoning, leading models to rely on language priors rather than true causal dependencies. To address this, the authors introduce Causal‑Plan‑Bench, a diagnostic suite covering four causal dimensions, and Causal‑Plan‑1M, a million‑scale corpus of explicit causal reasoning traces extracted from egocentric videos. Extensive experiments show that existing models perform poorly on these tasks, while a new model trained with a tailored recipe—Causal Planner based on Qwen3‑VL‑8B—achieves significant gains, demonstrating the feasibility of physically grounded causal reasoning.
By Zheng Lu, Mingqi Gao, Qinlei Xie, Wanqi Zhong, Hanwen Cui, Zirui Song, Lijie Wang, Chong Luo, Bei Liu, Yiming Li
HClimRep‑Ocean is a machine‑learning emulator that operates directly on the native unstructured mesh of the FESOM2 ocean model, trained on a 209‑year AWI‑CM3 control run and run without atmospheric forcing except at initialization. It shows strong skill for current forecasts at 30‑day lead times, outperforming all references, while temperature and salinity forecasts are best served by a damped‑anomaly persistence approach. In independent OceanBench testing, a reanalysis‑trained variant achieves the lowest RMSE against GLORYS reanalysis, demonstrating the competitiveness of the native‑mesh approach.
By Kacper Nowak, Aleksei Koldunov, Nikolay Koldunov, Savvas Melidonis, Ankit Patnala, Simon Grasse, Julius Polz, Christian Lessig, Martin Schultz, Thomas Jung
The paper introduces ECHO, a paired diagnostic benchmark for evaluating Chinese real‑time spoken dialogue systems on turn‑taking. ECHO pairs examples that share the same overlap transcript but differ in preceding multi‑turn context, requiring either Yield or Keep actions, and also includes off‑talk cases to test unnecessary yielding. Experiments on four speech systems reveal that three systems over‑yield, correctly keeping the floor on fewer than 13% of backchannels, while the fourth system shows a more balanced performance, illustrating that interruption‑only evaluation can overestimate turn‑taking reliability.
By Shuofeng Zhao, Hongwei Cai, Wenke Fan, Qingxiang Guo, Zhou Wang, Dawei Yang, Zhiyang Zhou, Yingxin Shang, Weixu Wang, Lin Yang, Shuran Zhou, Yang Song
The paper introduces scalable Graph Transformers for classifying healthy versus tumor epithelial cells in whole-slide images of cutaneous squamous cell carcinoma. By constructing a full‑WSI cell graph and incorporating morphological, texture, and neighboring cell class features, the proposed SGFormer and DIFFormer models outperform traditional image‑based methods, achieving balanced accuracies above 85% on single‑WSI tests and 83.6% on multi‑WSI evaluations. The study demonstrates that preserving tissue‑level context through graph representations improves classification of morphologically similar cell types.
By Lucas Sanc\'er\'e, No\'emie Moreau, Katarzyna Bozek
The paper presents EvalAgent, an AI assistant that automates agent evaluation by encoding domain expertise into evaluation skills such as procedural instructions, reusable code, and dynamic API retrieval. EvalAgent constructs a trace-based pipeline that outputs metrics, executable code, and reports, and is evaluated using a new meta-evaluation framework and AgentEvalBench. Results show that EvalAgent improves the Eval@1 metric from 17.5% to 65% and receives 79.5% human expert preference, while ablation studies confirm the importance of evaluation skills.
By Kang Zhou, Sangmin Woo, Haibo Ding, Kiran Ramnath, Subramanian Chidambaram, Aosong Feng, Vinayak Arannil, Muhyun Kim, Ishan Singh, Darren Wang, Zhichao Xu, Megha Gandhi, Nirmal Prabhu, Soumya Smruti Mishra, Smeet Dhakecha, Vivek Singh, Gouri Pandeshwar, Lin Lee Cheong
FluidRain is a lightweight video deraining model that leverages a divergence‑free rain flow field to guide Loop‑in‑Loop attention across scales and neighboring frames, eliminating the need for explicit motion alignment. By projecting estimated rain‑flow onto a divergence‑free subspace, the method steers window attention along rain streaks, enabling efficient temporal aggregation with only 0.80 M parameters. Experiments on four benchmarks demonstrate competitive performance against larger models, and the authors introduce a new RainSyn‑Gust dataset and a physics‑based no‑reference metric for evaluating real‑rain removal.
By Pu Wang, Yongcong Wang, Wenhao Li, Xiang Chen, Guangwei Gao, Jinshan Pan, Siyuan Yao, Shujun Fu, Zhuoran Zheng
FounRef is a training‑free method that refines frozen monocular foundation priors into dense metric depth by aligning them with sparse metric anchors. It validates anchors against the prior’s predictions, rejects misaligned ones, and applies a structure‑preserving solver to correct depth globally and locally while preserving fine geometry. The approach works out of the box on unseen cameras and scenes, achieving up to 24% lower depth error, 92% lower surface‑normal noise, and nearly 15× faster inference than a leading depth‑completion network.
By Dan Halperin, Mirko M\"ahlisch
ComplexSync is a diffusion-based framework that delivers real‑time, high‑fidelity lip synchronization even in complex scenarios. It uses a dual‑stream joint training strategy to prevent reference‑frame leakage, a distillation‑based acceleration for single‑step denoising that reaches over 70 FPS, and a relational alignment loss that incorporates structural priors from Vision Foundation Models to improve robustness. The authors also introduce the first benchmark for complex lip synchronization, featuring more than 200 challenging video sequences and specialized metrics, and show that ComplexSync outperforms existing methods on both standard and complex tasks.
By Jiaran Cai, Xingpei Ma, Shenneng Huang
PROVE is a black‑box hallucination detector for medical visual question answering that tailors its verification strategy to each question’s evidential structure. It classifies questions into three regimes, activates a subset of five operators per regime, and calibrates operator importance using deterministic question‑answer features to produce a risk score. On 8048 test samples across three medical VQA benchmarks and four state‑of‑the‑art vision‑language models, PROVE achieves an AUROC of 0.821, surpassing the best baseline by 0.159 with consistent improvements across all models and datasets.
By Keyang Zhou, Siyi Li, Zhongnan Shi, Qichao Ying, Wei Tang, Zhenxing Qian
Match4Annotate is a test‑time framework that transfers user‑specified annotations from a labeled ultrasound video to an unlabeled target video without requiring manual initialization. It uses a spatiotemporal implicit feature representation, a continuous implicit feature flow for alignment, and flow‑guided annotation transfer to unify sparse point and dense mask transfer. The method achieves state‑of‑the‑art performance on four clinical ultrasound datasets, outperforming dense feature‑matching baselines and one‑shot segmentation methods, and works without task‑specific training on a single consumer GPU.
By Zhuorui Zhang, Roger Pallar\`es-L\'opez, Praneeth Namburi, Brian W. Anthony
NNV3 is the latest version of the Neural Network Verification tool, a MATLAB framework for formally verifying deep learning models and learning‑enabled cyber‑physical systems. It builds on earlier NNV releases by adding new Star‑set members—ModelStar for weight perturbation, VolumeStar for video and 3D volumetric inputs, and GraphStar for graph neural networks—alongside a probabilistic reachability mode and FairNNV for fairness certification. The update also introduces benchmarks in malware detection, power‑system modeling, medical imaging, variable‑length time series, and action recognition, and provides unified documentation and tutorials.
By Anne M. Tumlin, Samuel Sasaki, Ben Wooding, Diego Manzanas Lopez, Muhammad Usama Zubair, Navid Hashemi, Hongchao Zhang, Waseem Abbas, Ipek Oguz, Meiyi Ma, Taylor T. Johnson
The paper introduces Speculative Evaluation, a method to reduce variance in evaluating stochastic large language models (LLMs) under a fixed rollout budget. It employs a Hierarchical Bayesian Neyman (HBN) policy that first runs a short uniform pilot, then pools task-level success counts via a hierarchical Bayesian model to compute posterior expectations of task-level sampling variances. Using these expectations, the method applies exact positive-integer Neyman allocation to allocate rollouts, and an asynchronous variant (HBN-async) speculatively executes continuations from partial pilot feedback to mitigate synchronization overhead. Across six checkpoints and 18 benchmark groups, Speculative Evaluation achieves 12.8%-33.6% lower variance compared to uniform allocation, outperforming empirical and independent Bayesian baselines, and demonstrates practical benefits in real-generation experiments.
By Qianli Shen, Xiang Li, Ruomeng Ding, Yanxi Chen, Daoyuan Chen, Yaliang Li
KeyGen is a framework that learns canonical 3D keypoints from point clouds to create structured, object‑centric representations for policy learning in robotic manipulation. By conditioning a visuomotor diffusion policy on these keypoints and object geometry, it predicts full manipulation trajectories that maintain geometric correspondence across different object instances. Experiments on a photorealistic simulation benchmark with three tasks show that KeyGen outperforms prior methods on both seen and unseen objects, scales with more demonstrations, remains robust to rescaling, and performs well in real‑world manipulation.
By Shuxin Cao, Liquan Wang, Masoud Moghani, Benjamin Joffe, Animesh Garg
The paper surveys 160 benchmarks from 2017‑2026 that evaluate predictive embodied intelligence, categorising them into policy suites, embodied agents, world‑model evaluation, and prediction‑to‑action bridges. It finds that most benchmarks are model‑agnostic, rarely compare Vision‑Language‑Action policies to world models, and seldom turn predictions into executed actions. The authors argue that the lack of benchmarks designed to directly test the closed‑loop advantage of world models prevents the field from answering whether such models truly improve robotic performance.
By Gaytri Jena, Kapil Wanaskar, Vinija Jain, Aman Chadha, Vasu Sharma, Amitava Das
GHOST-Q evaluates how post‑training quantization affects visual grounding in vision‑language models. The study compares three 8B VLM families across FP16, INT8, and NF4 precisions, pairing predictions to measure how compression redistributes grounding successes and failures. While most quantized variants maintain overall accuracy, several exhibit significant changes in hallucination‑sensitive conditions, and memory savings do not always translate to lower latency.
By Saim Rehman, Muhammad Shafique
The "Era by Eon" benchmark tests enterprise agents by presenting questions that specify answer rules and require code to compute answers from generated company data. While top models can answer most questions, the benchmark introduces eight new templates that rely on hidden facts not explicitly stated in any document, making the task harder. Evaluation of 12 agents shows that only the best agent correctly answers 18 of 24 attempts, with many questions remaining largely unsolved.
By Benjamin Gruenbaum, Doron Porat, Assaf Natanzon, Roy Zavida, Chen Dinachi, Or Itzahary
PFArena is a new benchmark for evaluating language models in protein modification tasks, featuring four controlled interfaces that span single‑mutant generation and multi‑mutant ranking. It incorporates varying levels of mutation fitness data to represent four research scenarios with different amounts of prior experimental context. The benchmark tests six protein language models, six large language models, and five LLM‑based agents, finding that PLMs excel at open‑ended single‑mutant generation while LLMs and agents perform best in multi‑mutant ranking when target‑specific data are available, yet all struggle as search space and mutation depth grow.
By Yawen Ouyang, Xinbo Zhang, Ziyuan Ma, Yixin Wu, Wenbin Liao, Feiran Zhang, Wenjie Li, Lihao Wang, Hao Wang, Xiaoqing Zheng, Xuefeng Yan, Lei Bai, Ya-Qin Zhang, Shuyi Zhang, Wei-Ying Ma, Dahua Lin, Bowen Zhou, Hao Zhou
The paper introduces a framework for Flow‑Matching Vision‑Language‑Action (VLA) models that allows independent adjustment of backbone depth, action expert depth, and denoising steps. Lightweight Exit Transformers are added at intermediate layers to enable early exits, and a KV Cache synthesis mechanism manages skipped layers so the action expert can exit deeper than the backbone. Experiments on SmolVLA and π0.5 across LIBERO and Meta‑World show that joint tuning of these compute axes reduces latency by 79.2 % and FLOPs by 31.8 %, while improving mean success rate by 5.6 %.
By Riccardo Andrea Izzo, Rimvydas Rubavicius, Gianluca Bardaro, Subramanian Ramamoorthy, Matteo Matteucci, Alessandro Suglia