arXiv Computation and Language
2d ago

CheXPoT: Post-Hoc Visual Verification and Correction of VLM-Generated Radiology Reports

CheXPoT is a post‑hoc visual verification framework that refines vision‑language model (VLM) generated chest radiograph reports by decomposing them into predicates, grounding each against DenseNet‑121 and Grad‑CAM evidence, and removing unsupported findings. On a held‑out set of 4,145 images, CheXPoT significantly improves precision and F1 scores across multiple CheXpert classes, boosting micro‑average precision from 19.2 % to 46.8 % and micro‑average F1 from 23.0 % to 49.4 %. The method also enhances performance for other VLM generators such as LLaVA‑Med and MAIRA‑2, demonstrating that inexpensive visual verification can correct hallucinations and improve report quality.

By Joey Guo, Rayan Ansari, Alaa Youssef