arXiv Computer Vision

Hawkeye: Reading Illegible Text with Vision Language Models

The paper introduces BSTRB, a benchmark of 8,398 noisy image samples with high‑fidelity annotations from 1,699 real‑world video sequences, designed to evaluate Vision‑Language Models (VLMs) on degraded text. It presents Hawkeye, a VLM fine‑tuned on BSTRB that reads and reasons about illegible text by using contextual priors and pixel‑level alignment. Hawkeye performs comparably to larger VLMs overall and surpasses them on text at or below 20 pixels, making it suitable for real‑time deployment in noisy egocentric settings.

arXiv Computer Vision
Sep 1

ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

ClearText-Video (CTVid) is a large-scale, scene-text-aware benchmark that examines text-centric video understanding under varying quality conditions. It comprises 4,639 real-world egocentric videos, over 550,000 frames, 1.6 million human-verified scene-text annotations, and more than 220,000 spatial/temporal question–answer pairs in Chinese and English. For each high-quality video, CTVid provides matched degraded- and restored-quality variants, enabling studies of Text-Centric Video Restoration and Multi-Quality VideoQA, and revealing that visual enhancement does not always improve textual fidelity or downstream reasoning.

By Jinlong Li, Jiaming Ding, Dingfu Lu, Malcolm Hsiu, Chuang Ke, Kangning Yang, Bochen Guan, Lan Fu, Jie Cai, Huiming Sun, Zibo Meng
arXiv Computer Vision
4d ago

FlexReader: Towards Adaptive Decoding Paths for Efficient Arbitrary-Length Scene Text Recognition

FlexReader is a scene text recognition model that supports multiple decoding strides, allowing a single model to balance accuracy and efficiency across various datasets. It introduces a Sub-string Perception module to extract target substrings from visual features and uses parallel decoding blocks to independently decode each character. During inference, an Adaptive Prefix Decoding strategy selects the optimal stride at each step based on confidence, enabling robust recognition of long, curved, or occluded text while speeding up decoding up to 6× compared to autoregressive methods.

By Zhibin Ma, Pengwen Dai, Zhigang Yao, Xugong Qin, Wenqi Ren, Xiaochun Cao
arXiv Computer Vision
Oct 12

TextVBench: Evaluating Text Rendering and Preservation in Video Generation

TextVBench is a new benchmark and evaluation framework for assessing how well video generation models render and preserve text. It introduces two tracks—TextVBench‑T2V for prompt‑specified text creation and maintenance, and TextVBench‑Ref2V for preserving text from reference images—using a multi‑target, multi‑instance spatiotemporal correspondence approach. The framework employs target‑conditioned candidate selection, supports multiple OCR regions per target, and separates spatial and temporal fidelity, summarizing results with a length‑weighted harmonic score and detailed diagnostics.

By Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang
arXiv Computer Vision
Oct 2

VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation

VTR-Bench is a new benchmark designed to evaluate how well video generation models render text within scenes. It includes 300 prompts across five real-world scenarios such as advertisements and scientific videos, and uses an automated pipeline with human alignment to assess text fidelity and scene/motion requirements. Experiments on 11 state‑of‑the‑art models show that even the best performer has a word error rate of 0.250, underscoring widespread challenges in visual text rendering.

By Yu Huang, Jungang Li, Zhiyuan Wang, Yonghua Hei, Song Dai, Jiayu Yang, Deyuan Liu, Xiang Zheng, Xiaoshuang Shi, Hao Cheng, Kaidi Xu
arXiv AI
Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu
arXiv Computer Vision
Sep 25

MEVL-STP: Multi-Encoder and Vision Language Model for Arbitrarily Shaped Scene Text Spotting

MEVL-STP introduces a two‑stage pipeline for spotting arbitrarily shaped scene text. The detection stage fuses features from six frozen vision encoders via a hierarchical Feature Pyramid Network and a Progressive Scale Expansion network to produce precise polygon masks. The recognition stage then crops these masks and feeds them to a fine‑tuned Qwen3‑VL‑8B‑Instruct model, achieving state‑of‑the‑art detection and end‑to‑end performance on CTW1500, Total‑Text, and ICDAR 2015 without synthetic pretraining.

By Aman Anand, Partha Pratim Roy, Shivakumara Palaiahnakote
arXiv AI
Sep 25

TOLA: Text-aware One-Step Latent Adaptation for Diffusion-based Text Image Super-Resolution

TOLA is a diffusion‑based text image super‑resolution method that eliminates iterative image‑text diffusion by using a one‑step latent adaptation framework. It employs a confidence‑weighted text conditioning module to build a reliable semantic condition and a lightweight latent residual correction module to fix structured residual errors, thereby preserving text fidelity. Experiments show TOLA outperforms existing diffusion‑based TSR methods, achieving at least 2.72 dB higher PSNR on the CTR‑TSR‑Test benchmark.

By Yike Xu, Yue Shi, Yong Guo, Jiezhang Cao
arXiv Computer Vision
Aug 28

ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction

ReViCo (Real Visual Correction) is a new benchmark that tests Vision Language Models (VLMs) on the task of correcting text errors in real‑world images, requiring deep understanding of visual text and its context. The study evaluates VLMs using both prompt‑based and targeted training approaches, revealing a significant performance gap between current models and humans. The results show that most VLMs struggle to accurately perceive visual text, leading to frequent correction mistakes, thereby underscoring the need for more robust, text‑aware VLMs.

By Bojun Zhang, Junhong Liang, Feifei Zhai, Fengxian Ji, Yu Zhou