Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Computer Vision
Sep 15

V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments

arXiv:2609.15683v1 Announce Type: new Abstract: While In-Context Learning (ICL) enables models to adapt from exemplars without parameter updates, multimodal ICL remains largely underexplored, particu...

By Ziqian Fan, Shibo Xu, Junjie Li, Xiangyu Zhao, Shengyuan Ding, Yifan Yang, Zhenjie Yang, Haodong Duan, Yue Zhou, Zhihang Zhong, Xue Yang
arXiv Computer Vision
Sep 15

Integrating Multi-view Multi-light Surface Reconstruction into Cultural Heritage Workflows

arXiv:2609.15833v1 Announce Type: new Abstract: Cultural heritage documentation increasingly relies on image-based 3D surface reconstruction, with photogrammetry software making such workflows access...

By Baptiste Brument, Robin Bruneau, Benjamin Coupry, Vincent Demoulin, Jean M\'elou, Antoine Laurent, Fabien Castan, Jean-Denis Durou, Lilian Calvet
arXiv AI
Sep 15

An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS

The paper introduces a modular correction framework for large language models that uses Activated LoRA adapters and a context-aware routing mechanism to mitigate harmful outputs. By allowing expert adapters to activate mid-sequence without invalidating the KV cache, the system achieves low-latency, targeted correction during generation. Experiments show improved alignment on safety benchmarks while maintaining task performance, presenting a lightweight, scalable approach to safer LLM deployments.

By Roberto Campbell, Momin Abbass, Muneeza Azmat, Michal Ulewicz, Raya Horesh, Kristjan Greenewald, Rog\'erio Abreu de Paula, Nathalie Baracaldo
arXiv Computer Vision
Sep 15

GorillaWatch: An Automated System for In-the-Wild Gorilla Re-Identification and Population Monitoring

arXiv:2512.07776v2 Announce Type: replace Abstract: Monitoring critically endangered western lowland gorillas is currently hampered by the immense manual effort required to re-identify individuals fr...

By Maximilian Schall, Felix Leonard Kn\"ofel, Noah Elias K\"onig, Jan Jonas Kubeler, Maximilian von Klinski, Joan Wilhelm Linnemann, Xiaoshi Liu, Iven Jelle Schlegelmilch, Ole Woyciniuk, Alexandra Schild, Dante Wasmuht, Magdalena Bermejo Espinet, German Illera Basas, Gerard de Melo
arXiv Computer Vision
Sep 15

Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language

arXiv:2603.02790v2 Announce Type: replace Abstract: Foundation models are changing the way we develop medical artificial intelligence. By learning broadly generalizable features across diverse data m...

By Michelle Stegeman (and on behalf of the UNICORN consortium), Lena Philipp (and on behalf of the UNICORN consortium), Fennie van der Graaf (and on behalf of the UNICORN consortium), Marina D'Amato (and on behalf of the UNICORN consortium), Cl\'ement Grisi (and on behalf of the UNICORN consortium), Luc Builtjes (and on behalf of the UNICORN consortium), Joeran S. Bosma (and on behalf of the UNICORN consortium), Judith Lefkes (and on behalf of the UNICORN consortium), Rianne A. Weber (and on behalf of the UNICORN consortium), James A. Meakin (and on behalf of the UNICORN consortium), Thomas Koopman (and on behalf of the UNICORN consortium), Anne Mickan (and on behalf of the UNICORN consortium), Mathias Prokop (and on behalf of the UNICORN consortium), Ewoud J. Smit (and on behalf of the UNICORN consortium), Fr\'ed\'erique Meeuwsen (and on behalf of the UNICORN consortium), Geert Litjens (and on behalf of the UNICORN consortium), Jeroen van der Laak (and on behalf of the UNICORN consortium), Bram van Ginneken (and on behalf of the UNICORN consortium), Maarten de Rooij (and on behalf of the UNICORN consortium), Henkjan Huisman (and on behalf of the UNICORN consortium), Colin Jacobs (and on behalf of the UNICORN consortium), Francesco Ciompi (and on behalf of the UNICORN consortium), Alessa Hering (and on behalf of the UNICORN consortium)