Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Machine Learning

Oct 8

Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting

arXiv:2603.11938v2 Announce Type: replace-cross Abstract: Structured radiology reporting promises faster, more consistent communication than free text, but automation remains difficult as models must...

By Chantal Pellegrini, Adrian Delchev, Ege \"Ozsoy, Nassir Navab, Matthias Keicher

llms rag multimodal benchmarks

arXiv Computation and Language

Oct 8

sk-bench: A Native-First Benchmark for Evaluating Large Language Models in Slovak

arXiv:2610.09152v1 Announce Type: new Abstract: Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation....

By Marek \v{S}uppa, Ivan Vykopal, Andrej Ridzik, Kristi\'an Sopkovi\v{c}, Nat\'alia K\v{n}a\v{z}ekov\'a, Jaroslav Kop\v{c}an, Miroslav Bl\v{s}t\'ak, Vikt\'oria Ondrejov\'a, Daniel Hl\'adek, Michal Gregor, Martin Tamajka, Mari\'an \v{S}imko

llms nlp benchmarks

arXiv Computation and Language

Oct 8

RunningTab: Direct Workspace Interaction with Environment-Side Tabs

arXiv:2610.10444v1 Announce Type: cross Abstract: Much knowledge work produces new deliverables from files a workspace already holds, and LLM agents are beginning to take such work over. Through dire...

By Jinheon Baek, Soyeong Jeong, Yumin Choi, Dongsu Han, Sung Ju Hwang

llms agents benchmarks

arXiv Computer Vision

Oct 8

PanoPed: Beyond Bounding Boxes for Sim-to-Real Panoramic Pedestrian Tracking

arXiv:2610.08826v1 Announce Type: new Abstract: Full-sphere panoramic cameras let fixed monitoring systems and mobile robots track people in every direction, but a planar bounding box does not fully...

By Qinfeng Zhu, Weiguang Zhao, Yunxi Jiang, Anh Nguyen, Lei Fan

robotics fine-tuning benchmarks

arXiv Computer Vision

Oct 8

S2Tok: Streaming 3D Gaussian Reconstruction with Persistent Spatial Tokens

arXiv:2610.08978v1 Announce Type: new Abstract: Streaming 3D reconstruction requires more than a sequence of geometric predictions: it requires a persistent scene state that can incorporate new evide...

By Fang Li, Jiraphon Yenphraphai, Quentin Herau, Depu Meng, Yihan Hu, Tianshuo Xu, Narendra Ahuja, Wei Zhan

llms benchmarks

arXiv Computer Vision

Oct 8

InstanceBench: Diagnosing Referential Reasoning and Target Identity in Referring Expression Segmentation

arXiv:2610.09478v1 Announce Type: new Abstract: Referring Expression Segmentation (RES) links natural-language descriptions to pixel-level object masks. Yet standard evaluation provides limited insig...

By Yuchen Li, Shaoyang Zhou, Yiran Wang, Ruiyi Deng, Haoyu Wang, Ziru Wei, Zhen Zhao, Luping Zhou

computer-vision benchmarks

arXiv Computer Vision

Oct 8

Scalable Patch-Level Self-Supervised Learning

arXiv:2610.10013v1 Announce Type: new Abstract: Self-supervised learning (SSL) at scale produces powerful visual representations. However, most scalable SSL methods rely on ad hoc combinations of mul...

By Maximilian Seitzer, Gabriele Trivigno, Anton\'in Vobeck\'y, Seungeun Yi, Maxime Oquab, Huy V. Vo, Oriane Sim\'eoni, Piotr Bojanowski

computer-vision benchmarks

arXiv Computer Vision

Oct 8

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

arXiv:2610.10390v1 Announce Type: new Abstract: Vision-language-action~(VLA) models have emerged as a promising paradigm for autonomous driving. However, existing VLA models still suffer from a funda...

By Xingtai Gui, Yucheng Zhou, Dongqian Guo, Jiahao Gong, Feiyang Tan, Jianbing Shen

agents robotics multimodal benchmarks safety

arXiv Computer Vision

Oct 8

MambaDSF: Multi-Scale SSM with Dilated Feature Fusion for Sonar Small Target Detection

arXiv:2605.24928v2 Announce Type: replace Abstract: Sonar imaging is the primary modality for underwater target detection, yet small targets remain difficult to detect due to insufficient pixel cover...

By Hui Lin, Jiayi Li, Jing Wang, Shenghui Rong

llms benchmarks safety

arXiv Computer Vision

Oct 8

HGPTrans: Hierarchical Graph-Pooling Transolver for Automotive Aerodynamic Drag Coefficient Prediction

arXiv:2609.31765v2 Announce Type: replace Abstract: Accurate and rapid prediction of the aerodynamic drag coefficient ($C_D$) is essential for vehicle design, particularly during early-stage design,...

By Bo Liu, Fengli Zhang, Qiuli Luo, Lianrui Nie, Wenjiang Wang

fine-tuning benchmarks

arXiv Computer Vision

Oct 8

EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning

arXiv:2605.01238v2 Announce Type: replace-cross Abstract: Engagement, which links to attentional, emotional, and cognitive dimensions, plays an important role in learning. In online and video-based l...

By Zikang Leng, Edan Eyal, Yingtian Shi, Jiaman He, Yaqi Liu, Thomas Pl\"otz

llms multimodal benchmarks

arXiv Machine Learning

Oct 8

Algorithmic Scratchpads and Curriculum Staging for Arithmetic Reasoning in Tiny Transformers

arXiv:2610.09003v1 Announce Type: new Abstract: Autoregressive Large Language Models (LLMs) frequently struggle with deterministic multi-step algorithmic tasks such as multi-digit multiplication and...

By Sourabh Kasliwal

llms rag benchmarks

arXiv Machine Learning

Oct 8

U-Space: Uncovering When and Why Uncertainty Arises in Language Models

arXiv:2610.09087v1 Announce Type: cross Abstract: Large language models are informing decisions with ever-higher stakes. As the consequences of their errors grow, a central question becomes harder to...

By Tobias Braun, Nils Loose, Alexander Herzog, Virginia Ceccatelli, Marcus Rohrbach, Thomas Eisenbarth, Lorenzo Cavallaro

llms benchmarks safety

arXiv Machine Learning

Oct 8

Understanding and Mitigating Token-Pruning-Induced Vulnerabilities in VLMs

arXiv:2610.09703v1 Announce Type: cross Abstract: Token-Pruning accelerates Vision-Language Models by removing redundant visual tokens, yet its safety implications remain underexplored. In this work,...

By Shuailong Wang, Xinyu Lyu, Shengming Yuan, Jingkuan Song, Heng Tao Shen, Lianli Gao

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

Oct 8

Dataset Pruning from First Principles: A Label-Free Linear Programming Approach

arXiv:2610.10347v1 Announce Type: cross Abstract: Dataset pruning reduces a large training set to a representative subset while preserving model performance. Existing geometry-based methods typically...

By Rodrigo Schuller, Francisco Ganacim

rag efficiency benchmarks

arXiv Machine Learning

Oct 8

Questioning the Questions: Sustaining Self-Evolution in Reasoning Models

arXiv:2610.04299v2 Announce Type: replace Abstract: Self-evolving reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper...

By Jinyuan Li, Chengsong Huang, Langlin Huang, Donghong Cai, Shiping Gao, Yuyi Yang, Jiaxin Huang

benchmarks

arXiv Machine Learning

Oct 8

TAVIS: A Benchmark for Egocentric Active Vision and Anticipatory Gaze in Imitation Learning

arXiv:2605.07943v2 Announce Type: replace-cross Abstract: Active vision -- where a policy controls its own gaze during manipulation -- has emerged as a key capability for imitation learning, with mul...

By Giacomo Spigler

rag diffusion robotics benchmarks

arXiv Machine Learning

Oct 8

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

arXiv:2610.06401v2 Announce Type: replace-cross Abstract: Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing tr...

By Mohamed Dhouib, Clement Elliker, Alexi Canesse, Ma\"el Jenny, Lucas-Andrei Thil, Mahammed El Sharkawy, Sonia Vanier, Elie Bursztein

llms agents efficiency benchmarks

arXiv Computer Vision

Oct 8

TopoCurve: Geometry-Aware Topology Reasoning via B\'ezier Curves in Autonomous Driving

arXiv:2610.09118v1 Announce Type: new Abstract: Topology reasoning jointly detects 3D lanes and traffic elements from multi-view images and infers their structural connectivity. Current methods model...

By Mihai Bogdan Deaconu, Laura Dio\c{s}an

agents robotics benchmarks safety

arXiv Computer Vision

Oct 8

Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models

arXiv:2610.09550v1 Announce Type: new Abstract: Vision-language models increasingly reason through crops, regions, and tool-produced observations. Yet an observation can influence the answer without...

By Huiyao Zhang, Jin Bai, Zilong Su, Rui Guo, Chaofan Qin, Jinze Lv, Wenhui Yu, Hongfei Wang, Ye Li

llms multimodal benchmarks

Get the daily briefing