arXiv AI
Jun 6
Personal AI Agent for Camera Roll VQA
arXiv:2606. 05275v1 Announce Type: cross Abstract: We study the personal camera roll visual question answering setting.
NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities
NoteVQA is a new benchmark that collects 252 real‑life visual questions from the Chinese image‑sharing platform Xiaohongshu, covering 12 topics and 7 user intents. Each question is paired with a concise expert reference and a human‑audited interleaved answer that blends text and visual evidence. The study evaluates VLMs on short‑answer correctness and interleaved answer quality using a new AgenticInterleave framework and a 12‑dimensional IVR‑12 rubric, finding that even state‑of‑the‑art models achieve only about 53% accuracy and lag behind human references in content quality.
Related stories
arXiv Computation and Language
Oct 16
VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents
VELD-QA is a large-scale benchmark for visual evidence grounding and question answering in long documents, comprising 15,000 human-authored question-answer pairs across 5,859 PDFs with 372,445 pages. The dataset includes detailed annotations of supporting pages, visual evidence elements, and descriptions of cited visual content, enabling separate evaluation of retrieval, localization, and interpretation. Experiments on nine vision-language models and various retrieval-augmented systems reveal significant gaps in evidence localization and visual interpretation, with oracle accuracy peaking at 81.7% and enrichment of visual-element annotations narrowing performance gaps by 17–24%.
arXiv Computer Vision
Sep 1
Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answering
arXiv:2604.07146v3 Announce Type: replace Abstract: Knowledge-based visual question answering (KB-VQA) requires vision-language models to understand images and use external knowledge, especially for...
arXiv Machine Learning
Jul 7
Incentivizing Vision Language Models to Search for Long Video Question Answering
arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.
arXiv Machine Learning
Sep 7
An Integrated Vision-and-Language Pretraining (VLP) and Visual Question Answering (VQA) model to Automate Nondestructive Evaluation Image Analysis
The paper presents ChatNDE Figure to Caption, an AI system that automates the interpretation of nondestructive evaluation (NDE) images. It combines a Vision‑and‑Language Pretraining (VLP) approach using ResNet50 for visual feature extraction and GPT‑2 for natural‑language captioning, evaluated with BLEU scores. Additionally, a Visual Question Answering (VQA) model is integrated to answer specific queries about the images, enhancing interactivity for field inspectors.
arXiv Machine Learning
Jun 15
Self-Evolving Visual Questioner
arXiv:2606. 13929v1 Announce Type: cross Abstract: Vision-language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non-trivial, visual-centric and grounded questions remains underexplored.
arXiv AI
Jul 28
Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
arXiv:2607. 22643v1 Announce Type: new Abstract: Multimodal retrieval-augmented generation (mRAG) aims to answer image-text queries with external knowledge, but most existing systems still retrieve directly from raw multimodal input over a flat evidence space.
arXiv Computer Vision
Oct 1
Composition, Not Conversation: VLMs Lose the Scene, Not the Thread
arXiv:2609.38368v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly reason over visual evidence that is cropped, segmented, retrieved, or revealed over time. Yet most VQA bench...
Hugging Face Trending Papers
Jul 23
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond a provided image to answer questions. KI-VQA involves multiple sub-problems -referring expression understanding, visual grounding, object recognition, knowledge retrieval, and reasoning-yet existing benchmarks typically report only end-task accuracy, obscuring where failures arise.
arXiv AI
Sep 10
Companion-style QA Assistance in Ego-Vision
BuddyVQA is a new benchmark for companion‑style question answering on egocentric streaming video, comprising 21.6K questions tied to 6K highlight moments across 1,012 long first‑person videos. It emphasizes two often overlooked aspects of daily first‑person QA: ego‑deictic expressions and interactively chained questions, requiring models to resolve visual pronouns and infer user intent within a long‑form streaming context. The authors propose MyBuddy, a multimodal chain‑of‑thought QA assistant that uses a question filter and multi‑level memory to efficiently retrieve visual and QA information, achieving significant performance gains on BuddyVQA and generalizing to other streaming and common video QA benchmarks.
arXiv AI
Jul 24
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
arXiv:2607. 21155v1 Announce Type: cross Abstract: Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond a provided image to answer questions.
arXiv Machine Learning
Jun 15
Pix2Fact: When Vision Is Not Enough -- Benchmarking Fine-Grained VQA with Web Verification on High-Resolution Real-World Scenes
arXiv:2602. 00593v4 Announce Type: replace-cross Abstract: Despite progress on general tasks, vision-language models (VLMs) still struggle with challenges that demand both fine-grained visual grounding and external knowledge, a synergy overlooked by existing benchmarks that evaluate these abilities in isolation.