Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Machine Learning
Sep 16

Stable by Construction: Variational Latent Markov Operators for Long-Horizon PDE Prediction

The paper introduces a variational framework called VAMO that incorporates latent Markov dynamics for neural PDE solvers, aiming to improve long‑horizon predictions by mitigating error accumulation. By representing physical states as latent distributions and evolving them through probabilistic transitions, the method aligns learned dynamics with a spectral geometry induced by structured Gaussian perturbations. Experiments on fluid‑dynamics benchmarks show that VAMO reduces error growth and enhances rollout stability compared to deterministic and noise‑injection baselines.

By Junyi Liao, Johann Guilleminot, Vahid Tarokh
arXiv AI
Sep 16

From Transient Prompts to Persistent Control: Scientific Poster Generation via Recursive Semantic-Geometric Contracts

The paper introduces PosterVisor, a framework that replaces transient prompts with persistent control for scientific poster generation. It uses an Orchestrator to compile rubrics into Semantic‑Geometric Contracts that bind claims, sources, visuals, budgets, and spatial commitments, and employs Recursive Contract Enforcement to dynamically trigger checks and prevent silent regressions. Implementations in HTML/CSS and PPTX show improved QA accuracy and higher human preference on benchmark datasets.

By Runze Li, Yukun Zhao, Can Xu, Yucheng Shen, Shuaiqiang Wang, Jianmin Wu, Lingyong Yan, Dawei Yin
arXiv AI
Sep 16

The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesis

The paper introduces MIMIC, a framework that uses executable code to generate rigorous reasoning data for large language models (LLMs). By converting algorithms into verifiable reasoning trajectories through narrative fusion, code-guided test synthesis, and dynamic code instrumentation, MIMIC creates a Code-Instrumented Reward (CIR) that supplies dense, high‑fidelity supervision for reinforcement learning. Models trained with MIMIC’s synthetic dataset show significant, consistent improvements in general reasoning, complex mathematics, and fine‑grained deterministic tasks.

By Jinyang Zhang, Weibin Liao, Keqin Bao, Sihang Li, Shaobo Wang, Muyang Ye, Hongxin Ding, Yue Fang, Tianyi Tang, Fei Huang, Kexin Yang, Xingzhang Ren, Dayiheng Liu
arXiv Computer Vision
Sep 16

TEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentation

TEDi is a Temporal memory-Enhanced and Denoising Transformer designed for surgical instrument segmentation. It introduces a query-level memory bank with a memory search enhancement encoder to incorporate discriminative representations from past frames, and a temporal consistency denoising module that builds a cross‑frame semantic anchor to stabilize predictions. Experiments on EndoVis 2017 and EndoVis 2018 show that TEDi outperforms existing state‑of‑the‑art methods, indicating its effectiveness for computer‑assisted surgery.

By Jiahong Yuan, Weiming Mi, Tao Zhang, Haoyin Zhou