Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Machine Learning
Oct 15

Exhaustive Canonical Skeletons Generation: A Formal Dataset for Decomposable Symbolic Regression

The paper introduces an exhaustive dataset of univariate canonical skeletons for neural symbolic regression, generated deterministically via a constrained context-free grammar. These skeletons serve as generic structural representations that eliminate redundant expressions and provide uniform operator distribution, thereby avoiding simplicity bias. The authors also present a sampling strategy for clean numerical data and benchmark 150 SR problems across three tiers, demonstrating challenges for current state‑of‑the‑art methods.

By Giorgio Morales, Anik\'o Ek\'art
arXiv Machine Learning
Oct 15

In-Context Batch Black-box Optimization via Flow Matching

In-Context Batch Black-box Optimization via Flow Matching introduces FMTR, an amortized method that learns a continuous, history-conditioned proposal distribution for variable-size batch sampling in expensive black-box optimization settings. FMTR is trained offline using conditional flow matching on Thompson-sampling argmaxes refined through a horizon-dependent Langevin pushforward map, enabling adaptive exploration-exploitation over the optimization horizon. At test time, FMTR generates batches without task-specific surrogate refitting or numerical acquisition optimization, achieving strong performance on synthetic and real-world benchmarks while significantly reducing batch selection computational cost.

By Quoc-Anh Hoang Nguyen, Xinyu Zhang, Daolang Huang, Julien Martinelli, Samuel Kaski
arXiv Machine Learning
Oct 15

NeuroDETR: Multiscale Discrete Spatiotemporal Tokenization for Asynchronous EEG Event Detection

NeuroDETR is a novel multiscale discrete spatiotemporal tokenization framework designed for asynchronous EEG event detection. It combines structured tokenization, event-aware feature refinement, and a sparse attention module called FOCAL to capture both brief and sustained neural responses while preserving spatial hierarchy across EEG channels. Experiments on eight diverse EEG datasets show that NeuroDETR achieves state‑of‑the‑art performance in jointly classifying events and estimating their onsets and offsets.

By Yaozhi Wen, Kun Zhao, Wei Wei, Haoyang Chen, Kangning Wang, Yi Sun, Shaowei Gu, Dingkun Liu, Shuang Qiu, Huiguang He
arXiv Computation and Language
Oct 15

ANCHOR: Assessing Naturalistic Constraint Hierarchies and Ordering in Repository-Grounded Agentic Coding

arXiv:2610.16859v2 Announce Type: replace Abstract: For repository-grounded coding agents, a functionally correct patch is not necessarily an acceptable one: developers also constrain how the task mu...

By Wenqiang Zhu, Xinping Lei, Zhiyi Lai, Zhengshuo Gong, Tianzhuang He, Songwei Yu, Weihao Li, Jingxuan Xu, Zihang Xu, Gang Wu, Zhongqiu Wang, Yuhui Bai, Jie Li, Zhao Liu, Junhao Fan, Yan Wu, Yanan Wu, Jiaheng Liu, Quan Gu, Ariel Deng
arXiv AI
Oct 15

ELECT: Settling Identity Before Rendering for Scalable Story Visualization

arXiv:2610.15555v1 Announce Type: cross Abstract: Training-free story visualization without reference images must preserve recurring entities across frames while scaling to long narratives. Existing...

By Mehrajul Abadin Miraj, Md. Mahmudul Hasan, Meherun Farzana, Aniket Joarder, Mahmudul Hasan, Abir Chakraborty Partha, Md. Ahasanul Alam, Redwan Ahmed Rizvee, Md. Fahim Arefin, Md Mahmudur Rahman, Md. Tanvir Alam, Md Mosaddek Khan