arXiv:2606. 06356v1 Announce Type: new Abstract: Multimodal generative models produce fluent outputs but remain unreliable when generation must respect structured, domain-specific, or safety-critical knowledge.
arXiv:2602.02140v2 Announce Type: replace Abstract: Recent advances in unified multimodal models (UMM) have demonstrated remarkable progress in both understanding and generation tasks. However, wheth...
arXiv:2608. 05000v1 Announce Type: cross Abstract: Vision offers a critical axis for advancing foundation models, driving a shift towards natively unified multimodal pretraining.
The paper introduces UniSandbox, a decoupled evaluation framework with controlled synthetic datasets, to study whether understanding informs generation in Unified Multimodal Models. Results show a notable understanding‑generation gap, especially in reasoning generation and knowledge transfer. Explicit Chain‑of‑Thought (CoT) in the understanding module bridges this gap, and self‑training can internalize CoT for implicit reasoning during generation; query‑based architectures also exhibit latent CoT‑like properties that aid knowledge transfer.
The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.
arXiv:2503. 06211v3 Announce Type: replace-cross Abstract: Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as audio and images while effectively leveraging that knowledge remains challenging.
UnifiedAttack introduces a benchmark for testing the safety of large multimodal models (LMMs) in tasks that combine text and image to produce harmful content. The benchmark focuses on the additional harm that arises from cross‑modal synergy and includes filtered multimodal samples and synthesized disinformation queries. A synergistic hijacking framework—comprising In‑Context Reskinning (ICR) and Cognitive Planning Injection (CPI)—is proposed to expose vulnerabilities, and extensive evaluations show that UnifiedAttack consistently bypasses current alignment defenses in state‑of‑the‑art architectures.
arXiv:2606. 05290v1 Announce Type: cross Abstract: Recent progress in generative modeling has made safety control a central challenge, yet existing approaches remain largely model-specific, requiring retraining or tailored interventions for each new architecture.
arXiv:2607. 00293v1 Announce Type: cross Abstract: Achieving true artificial general intelligence requires foundation models capable of integrating new modalities without forgetting prior knowledge.
WorldCast is a benchmark that tests multimodal generative models on multi-step world state prediction, requiring them to forecast successive future states from multimodal observations and diverse references. The study evaluates 12 models across 320 manually designed cases in eight categories, revealing that advanced models like GPT-Image-2.5 achieve high scores while open-source models lag behind, especially on complex context and reasoning tasks. The results underscore the importance of integrating dynamic planning, reasoning, and unified multimodal capabilities for realistic world simulation.
arXiv:2506. 20699v2 Announce Type: replace Abstract: Learning in non-stationary and multi-context environments requires more than ordinary within-task generalization.
The paper investigates how visual understanding and generation objectives interact within unified multimodal models (UMMs). At the representation level, each objective enriches the other, but forcing them through the same computation path can cause one to dominate; a task‑decoupled architecture mitigates this. At the task and system levels, the authors demonstrate bidirectional transfer between shared knowledge and superior performance of an end‑to‑end UMM over a planner–executor pipeline on complex tasks.
arXiv:2607. 27372v1 Announce Type: new Abstract: The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages.