Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv AI
Jul 16

A Hybrid Mamba for Audio-Visual Navigation

arXiv:2607. 13110v1 Announce Type: cross Abstract: Since the paradigm centered on convolutional neural networks and recurrent architectures was established in 2020, the fundamental backbone networks for audio-visual navigation have undergone no essential changes for more than five years, making them inadequate to support efficient representation of dynamic multimodal sequences.

By Yi Wang, Yinfeng Yu
arXiv AI
Jul 16

Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

arXiv:2607. 13125v1 Announce Type: cross Abstract: We introduce Boogu-Image-0.

By Guoxuan Chen, Chufeng Xiao, Haoran Yang, Siyue Xie, Binxiao Huang, Ming Zhang, Cheuk Him Chau, Xinyu Fu, Yingzhao Lian, Tom S. Y. Li, Jintao Lin, Bowen Dong, Zian Qian, Yuhao Liu, Yuxuan Hu, Weikang Shi, Bin Zou, Bowen Zheng, Haoxuan Che, Chang Chen, Yuyang He, Heyang Sun, Tianyu Huang, Chong Hou Choi, Cheng Gong, Han Shi, Haoli Bai, Xihui Liu, Hongsheng Li, Qifeng Chen, Chao Huang, Rui Liu, Chenyang Lei
arXiv Machine Learning
Jul 16

HEDGEHOG: Hierarchical Evaluation of Drug Generators Through Rigorous Filtration

arXiv:2607. 13155v1 Announce Type: new Abstract: Generative molecular models can support early drug discovery by proposing new candidate compounds de novo.

By Daria A. Ryabchenko (Ligand Pro, Moscow, Russia, Skolkovo Institute of Science and Technology, Artificial Intelligence Center, Moscow, Russia), Pavel Gurevich (Ligand Pro, Moscow, Russia, Skolkovo Institute of Science and Technology, Artificial Intelligence Center, Moscow, Russia), Shamil Kadyrov (Ligand Pro, Moscow, Russia), Daria Frolova (Ligand Pro, Moscow, Russia, Skolkovo Institute of Science and Technology, Artificial Intelligence Center, Moscow, Russia), Kseniia Fedisheva (Ligand Pro, Moscow, Russia), Sergei A. Nikolenko (Ligand Pro, Moscow, Russia), Alexander Shapeev (Ligand Pro, Moscow, Russia, Skolkovo Institute of Science and Technology, Artificial Intelligence Center, Moscow, Russia), Marina A. Pak (Ligand Pro, Moscow, Russia)
arXiv AI
Jul 16

Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs

arXiv:2601. 02023v2 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) increasingly utilize massive context windows as working memory for autonomous tasks, their reliability fluctuates significantly depending on how information is distributed in real-world corpora.

By Amirali Ebrahimzadeh, Seyyed M. Salili