arXiv AI

APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

arXiv Computer Vision
Sep 14

Unified Text-Image Generation with Weakness-Targeted Post-Training

The paper introduces a post‑training approach that enables a single inference process to transition from text reasoning to image synthesis, eliminating the need for explicit modality switching. Using the 14B BAGEL model, the authors demonstrate that targeted post‑training data and reward‑weighted training improve multimodal image generation across four independent T2I benchmarks. The study highlights the benefits of joint text‑image generation and strategic data selection for enhancing T2I performance.

By Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad
arXiv Computer Vision
Sep 7

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

WeAgent-MMGenEdit is a comprehensive framework for multimodal agentic image generation and editing that addresses the unreliability of current models when prompts require external world knowledge. It introduces a multimodal harness with persistent evidence management, a scalable data construction pipeline producing 23K supervised trajectories and 14.7K RL tasks, and a bilingual benchmark (WeBench-MMGenEdit) for knowledge-intensive generation and multi-image editing. Post‑training methods based on SFT and RL further refine the agent policy and image backend, enabling a 30B‑parameter policy to outperform similarly sized models and approach the performance of a 1T‑parameter agent.

By Hui Zhang, Zongkai Liu, Liqiang Niu, Juntao Liu, Han Li, Zhen Cao, Wenchao Chen, Chengduo Zhao, Fandong Meng
arXiv Computer Vision
Sep 25

AdaPilot: Towards Scene-Adaptive Policy Learning for Cross-Generator Text-to-Image Quality Optimization

AdaPilot introduces a scene-adaptive, cross-generator policy for optimizing text-to-image generation quality. By framing multi-turn image generation as a Markov Decision Process and using reinforcement learning, it decouples the policy from specific generator internals, incorporates scene-aware and process-level rewards, and achieves superior quality and generalization compared to baselines. Experiments demonstrate that a single AdaPilot policy can transfer zero‑shot to unseen generators while consistently improving performance across all evaluated models.

By Wenjin Liu, Fayuan Ke, Yue Lu, Zhe Cui, Anh Tuan Luu, Haoran Luo
arXiv Computer Vision
Aug 28

RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing

RubricRM introduces a pairwise generative reward modeling framework that generates an input‑specific rubric—comprising evaluation dimensions, weights, and scoring criteria—to score candidate images. The method is trained in two stages: supervised fine‑tuning to learn the rubric‑based scoring paradigm and GRPO to refine dimension‑level rewards. Experiments on text‑to‑image generation and instruction‑based image editing benchmarks demonstrate that RubricRM outperforms existing specialized reward models and competes with strong proprietary MLLM judges while using smaller backbones.

By Zijian Kan, Wei Wang, Long Luo, Bing Zhao, Xuan Ren, Weixu Qiao, Wenbo Li, Hu Wei, Lin Qu
arXiv AI
Oct 7

Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation

The paper introduces Diffusion On-Policy Context Distillation (D-OPCD), a method that internalizes the benefits of an agentic harness—such as memory, skills, and iterative prompt refinement—into the weights of a diffusion model. By treating the agent‑improved prompt as privileged context, D-OPCD distills harness knowledge into the model, raising direct‑generation scores from 60.52 to 65.09 across four benchmarks. This internalization allows the harness to shed saturated skills and continue evolving, enabling a second round of skill evolution that further improves performance by 1.83 points.

By Wenxuan Wang, Zekai Liu, Weinan Zhang, Yu Cheng, Yang Yang
arXiv AI
Sep 3

Multimodal Language Models as Text-to-Image Model Evaluators

Multimodal Language Models as Text-to-Image Model Evaluators presents MT2IE, a framework where a multimodal large language model generates evaluation prompts and scores images, achieving higher correlation with human judgment than prior metrics. MT2IE recovers official T2I model rankings using only 20 prompts—far fewer than traditional benchmarks—and adapts prompts to each model’s performance, maintaining informative scoring ranges. The approach demonstrates that dynamic, interactive evaluation can replace static benchmarks as T2I models improve.

By Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat, Koustuv Sinha, Melissa Hall, Amy Zhang, Michal Drozdzal, Adriana Romero-Soriano
Hugging Face Trending Papers
Aug 12

Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes.