arXiv AI

ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models

ReasonUni introduces a reasoning-augmented reconstructive post‑training framework for unified multimodal models (UMMs) that couples image‑to‑text understanding and text‑to‑image generation. The framework first reasons about the desired edit and its target state, then generates the target image conditioned on this reasoning output. A tree‑structured reinforcement learning method, TreeGRPO, jointly optimizes the reasoning and generation modules, yielding consistent performance gains across multiple benchmarks.

arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
arXiv AI
Aug 7

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

arXiv:2603. 08652v2 Announce Type: replace Abstract: Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning.

By Haodong Li, Chunmei Qing, Huanyu Zhang, Dongzhi Jiang, Yihang Zou, Hongbo Peng, Dingming Li, Yuhong Dai, ZePeng Lin, Juanxi Tian, Yi Zhou, Siqi Dai, Jingwei Wu, Pheng-Ann Heng
arXiv Computer Vision
Sep 14

Unified Text-Image Generation with Weakness-Targeted Post-Training

The paper introduces a post‑training approach that enables a single inference process to transition from text reasoning to image synthesis, eliminating the need for explicit modality switching. Using the 14B BAGEL model, the authors demonstrate that targeted post‑training data and reward‑weighted training improve multimodal image generation across four independent T2I benchmarks. The study highlights the benefits of joint text‑image generation and strategic data selection for enhancing T2I performance.

By Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad
arXiv AI
Jun 17

MathVis-Fine: Aligning Visual Supervision with Necessity via Progressive Dependency-Guided Training for Multimodal Mathematical Reasoning

arXiv:2606. 17888v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning has extended from purely linguistic domains to multimodal scenarios; however, existing approaches often treat visual inputs as homogeneous or auxiliary signals, failing to capture the intricate and sample-specific dependencies between text and images in mathematical problem-solving.

By Wanshi Xu, Haokun Zhao, Haidong Yuan, Songjun Cao, Long Ma
Hugging Face Trending Papers
6d ago

MERIT: Multimodal Evidence Reasoning over Image Tables

MERIT is a three‑stage framework that enables language models to reason over table images by first adapting a table‑specific encoder through reconstruction, localization, and color‑conditioned supervision. It then fine‑tunes the model with supervised evidence selection and reasoning, and finally refines the policy using reinforcement learning with distinct rewards for evidence correctness, calculation, final answers, and output format. The approach is evaluated on table question answering and fact verification, demonstrating the benefits of visual inputs and training components across diverse table structures and visual presentations.

arXiv AI
4d ago

MERIT: Multimodal Evidence Reasoning over Image Tables

MERIT (Multimodal Evidence Reasoning over Image Tables) is a three‑stage framework designed to enable language models to reason over table images, which preserve visual cues and structural details that serialized textual tables often lose. The approach first adapts a table‑specific encoder through reconstruction, localization, and color‑conditioned supervision to address visual evidence acquisition. It then fine‑tunes the model with supervised evidence selection and reasoning, followed by reinforcement learning that rewards evidence correctness, calculation, final answers, and output format, thereby improving retrieval and computational accuracy in table question answering and fact verification tasks.

By Haoyang Jiang, Zhuotong Fan, Mingyang Yi, Ju Fan, Lanqing Hu, Xiao Zan
arXiv Computer Vision
Sep 7

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

WeAgent-MMGenEdit is a comprehensive framework for multimodal agentic image generation and editing that addresses the unreliability of current models when prompts require external world knowledge. It introduces a multimodal harness with persistent evidence management, a scalable data construction pipeline producing 23K supervised trajectories and 14.7K RL tasks, and a bilingual benchmark (WeBench-MMGenEdit) for knowledge-intensive generation and multi-image editing. Post‑training methods based on SFT and RL further refine the agent policy and image backend, enabling a 30B‑parameter policy to outperform similarly sized models and approach the performance of a 1T‑parameter agent.

By Hui Zhang, Zongkai Liu, Liqiang Niu, Juntao Liu, Han Li, Zhen Cao, Wenchao Chen, Chengduo Zhao, Fandong Meng