arXiv AI
Benchmarking Vision-Language Models on Synapse and Neuron Proofreading in Connectomics
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 30
Benchmarking Vision-Language Models on Synapse Detection and Proofreading in Connectomics
The paper benchmarks vision‑language models (VLMs) on two key tasks in connectomics: synapse detection (presence and polarity) and proofreading (split and merge errors). It evaluates 21 models (19 open, 2 closed) under zero‑shot, few‑shot, and LoRA fine‑tuning, comparing them to specialist models on datasets built from public resources. While most VLMs perform at chance in zero‑shot settings, LoRA fine‑tuning with a few thousand labels brings open models to specialist performance, and adapted VLMs outperform specialists on unseen species for merge‑error detection.
arXiv AI
Aug 18
OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models
arXiv:2602. 18094v2 Announce Type: replace-cross Abstract: Existing Visual-Language Models (VLMs) have achieved significant progress by being trained on massive-scale datasets, typically under the assumption that data are independent and identically distributed (IID).
arXiv Computer Vision
Oct 8
Visual Evidence Under Cross-Examination: Evaluating and Controlling Decision-Level Evidence Use in Vision-Language Models
arXiv:2610.09550v1 Announce Type: new Abstract: Vision-language models increasingly reason through crops, regions, and tool-produced observations. Yet an observation can influence the answer without...
arXiv Computer Vision
Oct 27
What Remains OOD? OODConceptBench for Open-World Concept Recognition in Vision-Language Models
arXiv:2610.33564v1 Announce Type: new Abstract: In the era of large-scale pretrained models, zero-shot generalization has increasingly blurred the boundary between known and unknown concepts. However...
arXiv AI
Oct 27
FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs
The paper introduces FOCUS, a cross‑dataset benchmark for retinal fundus models that evaluates vision‑only encoders, vision‑language dual encoders, and multimodal large language models across ten public datasets. It measures ranking performance, calibration, subgroup disparities, and image‑quality robustness for tasks such as diabetic retinopathy and glaucomatous optic neuropathy. Results show no single model family dominates, with vision‑only encoders ranking highest on average, while fine‑tuning improves in‑domain performance but varies in external transfer.
arXiv Machine Learning
Jun 30
DataComp-VLM: Improved Open Datasets for Vision-Language Models
arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.