← Back to all news

Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Computer Vision
Sep 23

Annual Earth-observation embeddings encode wildfire disturbance and support simplified burned area mapping

Annual Earth‑observation embeddings, specifically Tessera, can encode wildfire disturbance signals well enough to map burned areas without needing curated fire‑specific imagery or dense time‑series analysis. In tests, linear models using a single Tessera embedding matched or outperformed paired pre‑ and post‑fire HLS imagery and post‑fire imagery alone, achieving high F1 scores for burn‑scar delineation and regional mapping. The approach successfully mapped all same‑year fires in benchmark scenes, recovered 97% of California burned area without California training data, transferred to European fires with high accuracy, and even estimated ignition timing within a 13‑day error margin. "whyItMatters":"The study demonstrates that pre‑trained annual embeddings can simplify and scale burned‑area mapping, reducing reliance on dense time‑series data and enabling more efficient wildfire monitoring."

By Jovana Knezevic, Clement Atzberger, Zhengpeng Feng, Adam F. A. Pellegrini, Srinivasan Keshav, David Coomes
ragcomputer-visionbenchmarks
More like this →
arXiv Computer Vision
Sep 23

Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenes

Metric-Bench introduces a new benchmark for Vision‑Language Models (VLMs) that focuses on metric‑spatial reasoning in indoor scenes by using in‑image reference objects with known dimensions. The accompanying MetricReasoner fine‑tuning recipe employs structured prompts and numerical rewards to implicitly learn 2D‑to‑3D mapping without camera intrinsics. Experiments show that this approach improves spatial metric understanding by 43.1 % over existing models and boosts downstream embodied tasks, while also delivering gains on general VLM benchmarks.

By Yuling Xi, Haokai Zhang, Muzhi Zhu, Hao Zhong, Zongze Du, Hengyu Zhao, Chenchen Jing, Yufei Yin, Bin Qin, Yongjie Yang, Zhenbo Luo, Hao Chen, Chunhua Shen
llmsagentsroboticsfine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Sep 23

MIAR: Medical Image Super-Resolution With Autoregressive Modeling

MIAR introduces a multi‑scale autoregressive framework for medical image super‑resolution, treating the task as a conditional, progressive next‑scale prediction. It incorporates a Scale‑Adaptive Structural Decoder to preserve structural fidelity and uses a hierarchical beam search during inference to reduce recursive error accumulation. Experiments show MIAR outperforms existing methods, achieving a 7.86% MUSIQ improvement and a 2.02× speedup over diffusion‑based approaches.

By Fang Li, Yinglong Li, Hongyu Wu, Yang Gao, Minwei Zhao, Aimin Hao
diffusionefficiencybenchmarks
More like this →
arXiv AI
Sep 23

World State Generator

arXiv:2609.24744v1 Announce Type: new Abstract: Language agents solve complex tasks through plans and actions. A single step the world refuses puts the goal out of reach, and what the agent does next...

By Sungheon Jeong, Sanggeon Yun, Ryozo Masukawa, Haleh Alimohamadi, Mahdi Imani, Mohsen Imani
llmsagentsbenchmarkssafety
More like this →