← Back to all news
arXiv Computation and Language By Kartik Jangra, Atharva Gawane, Siddharth Pulipaka, Hemachandran Karah, Raj Dabre

Learning to Read Braille: Benchmarking and Improving Braille Recognition Capabilities of VLMs

Read the original on arXiv Computation and Language →

The paper introduces BrailleBench, a benchmark of 259 real-world Grade‑1 and Grade‑2 Braille images, and evaluates various vision‑language models (VLMs) on the task of transcribing Braille from images. It finds that general‑purpose VLMs perform poorly, while tool‑assisted systems do better. By training a 2B VLM on a synthetic Braille generation pipeline, the authors achieve 87% exact match on Grade‑1 and 61% on Grade‑2 BrailleBench, surpassing other evaluated systems, but also demonstrate that high synthetic validation accuracy does not translate to real‑world performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computer Vision
Sep 3

Test-Time Logit Prompting for Source-Free Missing Modality Adaptation

The paper introduces Test-Time Logit Prompting (TLP), a lightweight framework that adapts vision-language models to missing-modality inputs without accessing source training data. TLP optimizes logit prompts using uncertainty-aware adjustments and modality-complete consistency regularization, thereby maintaining prediction confidence and semantic consistency. Experiments on various benchmarks show that TLP improves recognition performance by up to 8% while requiring only a few hundred tunable parameters and minimal test-time optimization steps.

By Taixi Chen, Nancy Guo
llmsmultimodalbenchmarkssafety
More like this →
arXiv Computer Vision
Oct 19

VISTA: A Multimodal Egocentric Benchmark for Goal-Oriented Perception and Assistance to Blind and Low-Vision Users

VISTA (Visually Impaired Scene and Task Annotations) is a multimodal egocentric benchmark designed to evaluate goal‑oriented assistance for blind and low‑vision users. It comprises about 1,000 recordings from Project Aria glasses, each providing six synchronized sensor streams and three types of annotations across ten task categories, from object localization to hazard detection. The benchmark tests vision‑language models in zero‑shot settings and introduces a lightweight multimodal baseline that integrates all six modalities, revealing that current models can describe scenes but struggle to generate effective assistive guidance.

By No\'e Mosca, Mohamed Rayan Barhdadi, Muhammad Hamza, Joshua Yeh, Suryodaya Bikram Shahi, Zhiyan Li, Selim-Antoine Lali, Weiyue Li, Hasan Kurban, Erchin Serpedin, Gang Luo, Yue Zhang, Minghan Li, Mengyu Wang
llmsnlpmultimodalbenchmarks
More like this →
arXiv AI
Oct 27

FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs

The paper introduces FOCUS, a cross‑dataset benchmark for retinal fundus models that evaluates vision‑only encoders, vision‑language dual encoders, and multimodal large language models across ten public datasets. It measures ranking performance, calibration, subgroup disparities, and image‑quality robustness for tasks such as diabetic retinopathy and glaucomatous optic neuropathy. Results show no single model family dominates, with vision‑only encoders ranking highest on average, while fine‑tuning improves in‑domain performance but varies in external transfer.

By David Restrepo, Chenwei Wu, Luis Filipe Nakayama, Miguel L. Martins, Stergios Christodoulidis, Maria Vakalopoulou, Enzo Ferrante
llmsfine-tuningmultimodalbenchmarkssafety
More like this →