The Flow has not summarised this story yet — read it at arXiv Computation and Language.
This paper introduces Sense-Aligned Grounding Evaluation (SAGE), a unified 10-candidate image-selection benchmark that reformulates VWSD, VerSe, and WikiDiverse into a shared term-centric protocol cov...
arXiv:2607. 16727v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) suffer from error snowballing: a single incorrect inference early in a chainof-thought (CoT) trace corrupts all downstream reasoning.
UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.
arXiv:2608. 04726v1 Announce Type: new Abstract: Multimodal large language models increasingly reason over screenshots and documents where the task itself may be written in pixels.
WorldGenBench is a benchmark that tests text‑to‑image models on their ability to combine factual grounding with contextual reasoning. It introduces the Knowledge Checklist Score to evaluate whether generated images reflect both explicit and implicit semantic elements. The study finds that while some proprietary models perform well, most open‑source systems struggle, especially with culturally diverse and scientific content, revealing common failure modes such as semantic hallucination and underrepresentation of non‑Western knowledge.
arXiv:2608.30247v1 Announce Type: new Abstract: Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but of...