arXiv Computer Vision August 24, 2026 By Yun Li, Biao Yang, Peixi Wu, Yunhao Zhou, Mingzhou Jiang, Wei Yuan, Fan Yang, Wenwu Ou

Generative Embedding Benchmark: How Much Information Survives in a Dense Embedding?

Read the original on arXiv Computer Vision →

The paper introduces the Generative Embedding Benchmark (GEB), which evaluates how much content from an embedding can be recovered by a decoder that only has access to the frozen embedding and a question, without the original image or intermediate features. GEB uses a curated visual‑question‑answering dataset with 1,800 development and 900 test items covering natural images, scene text, and visual documents. Experiments on seven public embedding models show that visual‑only scores range from 28.25 to 33.21, while joint image‑question encoding boosts scores up to 65.56, revealing that generative readout uncovers information bottlenecks not captured by traditional separability‑based benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Machine Learning

Jul 9

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.

By Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez, Daniel DeAlcala, Aythami Morales, Ruben Tolosana, Oscar Delgado, Alvaro Ortigosa, Javier Ortega-Garcia

llms nlp fine-tuning multimodal

arXiv AI

Jun 16

VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

arXiv:2606. 16092v1 Announce Type: cross Abstract: Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements.

By Young Rok Jang, Hyesoo Kong, Kyunghwan An, Jae Sub Huh, Gyeonghun Kim, Stanley Jungkyu Choi

llms fine-tuning multimodal

arXiv AI

Sep 30

ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression

arXiv:2609.37225v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have shown strong potential for universal multimodal representation learning. However, existing methods eith...

By Zijing Cai, Yuzhe Wang, Jingxian Zhu, Fengbin Zhu, Richang Hong

llms rag multimodal

Hugging Face Trending Papers

Jul 8

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents. Although Vision-Language Models (VLMs) have shown remarkable performance in text-vision tasks, their robustness and transferability to different document domains remains underexplored.

llms nlp fine-tuning multimodal

arXiv Machine Learning

Sep 22

Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation

arXiv:2609.22162v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves the factuality of large language models (LLMs) and vision-language models (VLMs) by grounding generatio...

By Can Peng, Yu Liu, Yingyu Yang, Anjie Le, Yuyuan Liu, Qianye Yang, J. Alison Noble

llms rag nlp multimodal benchmarks

arXiv Computer Vision

Sep 16

Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models

The paper introduces three new vision‑centric evaluation benchmarks—temporal frame retrieval, video future prediction, and causal memory distortion—to assess visual question answering in large video models. Unlike traditional benchmarks that rely on text-based multiple choice questions, these tasks require models to reason directly from visual inputs. The authors find that current state‑of‑the‑art models struggle with visual queries, highlighting a gap in visual understanding that future research should address.

By Rwiddhi Chakraborty (Oliver), Yinong (Oliver), Wang, Cheng Zhang, Fan Bai, Zhuoran You, Michael Kampffmeyer, Yong Jae Lee, Fernando De la Torre, Robert Jenssen

nlp benchmarks

One email a morning, machine-written

Related stories

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression

Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering

Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation

Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models