arXiv AI By S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma

How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning

Read the original on arXiv AI →

Bio‑VQA is a new, large-scale visual question answering benchmark for biology, comprising 1.86 million questions over 217 k images from 11 datasets. It tests four key aspects of biological visual understanding—taxonomic reasoning, attribute grounding, spatial reasoning, and individual re‑identification—using a graded scoring system that rewards near‑misses more than distant ones. Evaluation of 18 open‑weight vision‑language models shows that general visual capabilities do not transfer well to biological tasks, with markedly lower accuracy on counting, taxonomy, trait measurement, and re‑identification compared to everyday object benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 12

Can Edge-Deployable Vision-Language Models Identify Species?

arXiv:2609. 11916v1 Announce Type: new Abstract: Camera traps often run in the field on edge hardware with limited or no connectivity, making small, locally-deployable vision-language models (VLMs) -- not frontier-scale ones -- the practically relevant class to evaluate for species identification.

By William Zhou, Mayukha Siripuram, Xiao Yan, Ziqi Liu, Yi Ding
Hugging Face Trending Papers
Aug 9

TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases

To address this gap, we introduce TomaMMU, a large-scale Tomato leaf disease MultiModal Understanding dataset, alongside TomaBench, a benchmark for evaluating VLMs on tomato disease understanding. TomaMMU comprises 28,808 high-quality images spanning 15 categories and 213,119 human-annotated visual question-answer pairs, generated through a three-stage pipeline comprising Data Collection, Human Annotation, and Question-Answer Generation.