arXiv:2503. 07265v4 Announce Type: replace-cross Abstract: Text-to-Image (T2I) models are capable of generating high-quality artistic creations and visual content.
WorldGenBench is a benchmark that tests text‑to‑image models on their ability to combine factual grounding with contextual reasoning. It introduces the Knowledge Checklist Score to evaluate whether generated images reflect both explicit and implicit semantic elements. The study finds that while some proprietary models perform well, most open‑source systems struggle, especially with culturally diverse and scientific content, revealing common failure modes such as semantic hallucination and underrepresentation of non‑Western knowledge.
arXiv:2607. 15216v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) often introduce errors when generating image captions, resulting in misaligned image-text pairs.
arXiv:2606. 10620v1 Announce Type: cross Abstract: Image generation models now produce high-quality static images, yet their ability to represent how a visual world changes over time remains poorly understood.
The paper introduces the Semantic Drift Protocol (SDP), a Telephone Game-inspired method to evaluate how well unified models preserve meaning when alternating between image-to-text (I2T) and text-to-image (T2I) generation over multiple generations. It defines Mean Cumulative Drift (MCD) and Multi-Generation GenEval (MGG) as metrics for semantic retention, and presents a new benchmark of 400 image‑text pairs from NoCaps and DOCCI to stress-test models beyond COCO. Applying SDP to seven models shows that models with strong single‑pass scores can still suffer severe semantic drift, revealing catastrophic failure modes that isolated benchmarks miss.
arXiv:2601. 22725v4 Announce Type: replace-cross Abstract: Recent advances in diffusion models have significantly elevated the visual fidelity of Virtual Try-On (VTON) systems, yet reliable evaluation remains a persistent bottleneck.
Vision2Code is a new benchmark for image-to-code generation that covers 15 domains and 539 test examples without relying on reference source code. It evaluates generated programs by executing them and comparing rendered outputs to the original images using domain‑specific rubrics, yielding scores that correlate better with human judgments than generic metrics. The study shows domain variability in model performance and demonstrates that the scoring method can improve fine‑tuning of models like Qwen3.5‑9B and Phi‑4‑MM via rejection sampling.
The paper introduces Auto-Comp, a fully automated, concept-driven pipeline that generates photorealistic compositional benchmarks for vision‑language models. Auto‑Comp creates paired Minimal and Contextual samples for each concept, enabling isolation of core binding abilities from visio‑linguistic complexity. Evaluations across 25 models reveal consistent failures in attribute and relational binding, with context helping relational tasks but hindering attribute tasks due to visual clutter.
Orbia is a new benchmark that systematically evaluates 3D consistency in interactive video world models. It organizes evaluation into three dimensions—input preservation, generated‑content persistence, and 3D self‑consistency—using controlled exploration and revisit protocols. The benchmark covers 720 instances across indoor and outdoor scenes and tests 27 different models, providing a comprehensive framework for diagnosing and improving 3D consistency.
arXiv:2606.03401v2 Announce Type: replace Abstract: Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated conten...
arXiv:2609.06245v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) perform strongly on general visual understanding tasks such as visual question answering, yet they often str...
arXiv:2606. 28406v1 Announce Type: new Abstract: Text-to-image and multimodal generative models are increasingly used to produce scientific figures such as mechanism diagrams, experimental-design schematics, conceptual frameworks, and graphical abstracts.