arXiv AI
Jun 6
Personal AI Agent for Camera Roll VQA
arXiv:2606. 05275v1 Announce Type: cross Abstract: We study the personal camera roll visual question answering setting.
NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities
NoteVQA is a new benchmark that collects 252 real‑life visual questions from the Chinese image‑sharing platform Xiaohongshu, covering 12 topics and 7 user intents. Each question is paired with a concise expert reference and a human‑audited interleaved answer that blends text and visual evidence. The study evaluates VLMs on short‑answer correctness and interleaved answer quality using a new AgenticInterleave framework and a 12‑dimensional IVR‑12 rubric, finding that even state‑of‑the‑art models achieve only about 53% accuracy and lag behind human references in content quality.
Related stories
arXiv Computer Vision
Sep 1
Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answering
arXiv:2604.07146v3 Announce Type: replace Abstract: Knowledge-based visual question answering (KB-VQA) requires vision-language models to understand images and use external knowledge, especially for...
arXiv Machine Learning
Jul 7
Incentivizing Vision Language Models to Search for Long Video Question Answering
arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.
arXiv Machine Learning
Sep 7
An Integrated Vision-and-Language Pretraining (VLP) and Visual Question Answering (VQA) model to Automate Nondestructive Evaluation Image Analysis
The paper presents ChatNDE Figure to Caption, an AI system that automates the interpretation of nondestructive evaluation (NDE) images. It combines a Vision‑and‑Language Pretraining (VLP) approach using ResNet50 for visual feature extraction and GPT‑2 for natural‑language captioning, evaluated with BLEU scores. Additionally, a Visual Question Answering (VQA) model is integrated to answer specific queries about the images, enhancing interactivity for field inspectors.
arXiv Machine Learning
Jun 15
Self-Evolving Visual Questioner
arXiv:2606. 13929v1 Announce Type: cross Abstract: Vision-language models (VLMs) are typically trained as passive answerers, while their ability to actively ask diverse, non-trivial, visual-centric and grounded questions remains underexplored.
arXiv AI
Jul 28
Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
arXiv:2607. 22643v1 Announce Type: new Abstract: Multimodal retrieval-augmented generation (mRAG) aims to answer image-text queries with external knowledge, but most existing systems still retrieve directly from raw multimodal input over a flat evidence space.
Hugging Face Trending Papers
Jul 23
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond a provided image to answer questions. KI-VQA involves multiple sub-problems -referring expression understanding, visual grounding, object recognition, knowledge retrieval, and reasoning-yet existing benchmarks typically report only end-task accuracy, obscuring where failures arise.
arXiv AI
Sep 10
Companion-style QA Assistance in Ego-Vision
BuddyVQA is a new benchmark for companion‑style question answering on egocentric streaming video, comprising 21.6K questions tied to 6K highlight moments across 1,012 long first‑person videos. It emphasizes two often overlooked aspects of daily first‑person QA: ego‑deictic expressions and interactively chained questions, requiring models to resolve visual pronouns and infer user intent within a long‑form streaming context. The authors propose MyBuddy, a multimodal chain‑of‑thought QA assistant that uses a question filter and multi‑level memory to efficiently retrieve visual and QA information, achieving significant performance gains on BuddyVQA and generalizing to other streaming and common video QA benchmarks.
arXiv AI
Jul 24
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
arXiv:2607. 21155v1 Announce Type: cross Abstract: Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond a provided image to answer questions.
arXiv Machine Learning
Jun 15
Pix2Fact: When Vision Is Not Enough -- Benchmarking Fine-Grained VQA with Web Verification on High-Resolution Real-World Scenes
arXiv:2602. 00593v4 Announce Type: replace-cross Abstract: Despite progress on general tasks, vision-language models (VLMs) still struggle with challenges that demand both fine-grained visual grounding and external knowledge, a synergy overlooked by existing benchmarks that evaluate these abilities in isolation.
arXiv Computer Vision
Aug 24
Generative Embedding Benchmark: How Much Information Survives in a Dense Embedding?
The paper introduces the Generative Embedding Benchmark (GEB), which evaluates how much content from an embedding can be recovered by a decoder that only has access to the frozen embedding and a question, without the original image or intermediate features. GEB uses a curated visual‑question‑answering dataset with 1,800 development and 900 test items covering natural images, scene text, and visual documents. Experiments on seven public embedding models show that visual‑only scores range from 28.25 to 33.21, while joint image‑question encoding boosts scores up to 65.56, revealing that generative readout uncovers information bottlenecks not captured by traditional separability‑based benchmarks.
arXiv AI
Jun 16
VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA
arXiv:2606. 16092v1 Announce Type: cross Abstract: Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements.