Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv AI
Oct 15

Graph-free Retrieval with Graph-Modified Embeddings

arXiv:2610.18410v1 Announce Type: cross Abstract: Dense vector retrieval can miss evidence connected through intermediate entities, while direct graph retrieval requires access to the graph at query...

By Federica Santi (Cognee Inc), Vasilije Markovi\'{c} (Cognee Inc), Daniel Scott (Cognee Inc), Lazar Obradovi\'{c} (Cognee Inc)
arXiv Machine Learning
Oct 15

ChemArgus: Automated Scientific Diagnostics for Lab-Grade Multimodal Chemistry Reasoning: A Frontier Benchmark

arXiv:2610.20012v1 Announce Type: new Abstract: Chemistry evaluation has moved past multiple-choice probes toward open-ended expert problems, yet the benchmarks that define the current frontier still...

By Hanyao Guo, Mo Cui, Zihan Tan, Leixin Sun, Yichun Wang, Yinan Shen, Guojun Zhu, Jia Jiang, Erhao Chen, Zui Chen, Wenzhe Gao, Zhihao Yu, Qijun Wang, Yetian Zhu, Jiande Chen, Chuhang Pei, Yao Jiang, Yichen Fan, Pinhan Wang, Xiaorui Pan, Qianyi Zhu, Yang Liu, Mang Ye, Guancheng Wan