The paper proposes a sketch-based visual grounding framework that converts text embeddings into topographic sketches to enhance visualization, explainability, and information retrieval. It evaluates the method on the MS-COCO dataset, finding that while SBERT outperforms the sketch-based approach in top‑1 retrieval, the two methods achieve comparable top‑5 results, and a cascaded combination improves performance further. The study suggests that sketch-based representations could be valuable for retrieval tasks if trained directly on text.
arXiv:2606. 17646v1 Announce Type: cross Abstract: Saliency map visualizations explain image-based AI predictions by pointing to regions, but these are often unintuitive and semantically unclear, leaving an interpretability gap.
By Wencan Zhang, Mario Michelessa, Xuejun Zhao, Brian Y. Lim
arXiv:2606. 24849v1 Announce Type: cross Abstract: Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved.
By Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun
Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream.
Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from document collections where evidence is distributed across t...
arXiv:2604.22875v3 Announce Type: replace-cross
Abstract: When answering questions about images, humans naturally point, label, and draw to explain their reasoning. In contrast, modern vision-languag...
By Brandon Collins, Logan Bolton, Hung Huy Nguyen, Mohammad Reza Taesiri, Trung Bui, Anh Totti Nguyen
arXiv:2608.15698v2 Announce Type: replace
Abstract: Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from docume...
By Chunyi Peng, Zhipeng Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Sen Mei, Yubo Sun, Yongheng Zhang, Jie Zhou, Yu Gu, Ge Yu, Maosong Sun
arXiv:2607. 12678v1 Announce Type: cross Abstract: Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and semantic cues for intelligent design understanding.
By Yan Gong, Bohao Li, Bowen Du, Junchen Ye
The paper introduces ExpArt-KG, a knowledge graph tailored to the artwork domain, and a retrieval‑augmented generation framework that alternates between generating answers and retrieving relevant facts from the graph. By using a correctness judgment to guide the search, the method efficiently gathers the necessary factual information, improving the detail of image explanations while reducing external knowledge retrieval costs. Experimental results demonstrate that the approach maintains generation quality comparable to fixed‑iteration methods.
By Yuta Kato, Shintaro Ozaki, Kazuki Hayashi, Yusuke Sakai, Hidetaka Kamigaito, Katsuhiko Hayashi, Taro Watanabe
arXiv:2603.26737v2 Announce Type: replace-cross
Abstract: Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual acc...
By Guangfu Guo, Xiaoqian Lu, Yue Feng, Mingming Sun
arXiv:2510. 04514v3 Announce Type: replace Abstract: Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts-those requiring precise visual interpretation rather than relying on textual shortcuts.
By Rachneet Kaur, Nishan Srishankar, Zhen Zeng, Sumitra Ganesh, Manuela Veloso
arXiv:2602. 02465v2 Announce Type: replace Abstract: Frontier models are transitioning from multimodal large language models (MLLMs) that merely ingest visual information to unified multimodal models (UMMs) capable of native interleaved generation.
By Jana Zeller, Thadd\"aus Wiedemer, Fanfei Li, Thomas Klein, Prasanna Mayilvahanan, Matthias Bethge, Felix Wichmann, Ryan Cotterell, Wieland Brendel