arXiv Computer Vision

ReART: Reference-Guided Retrieval and Refinement for Emotion-Aware Art Generation

arXiv AI
Sep 16

Text-Driven Artistic Staging: 3D Posing, Lighting, and Camera References from Paintings

The paper presents a method for generating 3D staging—human poses, lighting, and camera setup—directly from affective textual descriptions. It builds a dataset of 11,911 text–staging pairs derived from 2,328 figurative paintings, reconstructing SMPL bodies, estimating illumination, and recovering camera parameters. A flow‑matching transformer is trained to produce variable‑size scenes and multiple staging alternatives, achieving a 32.2% retrieval R@1 on held‑out prompts, outperforming a CLIP‑based baseline.

By Yunge Wen
arXiv Computer Vision
Oct 12

What We Talk About When We Talk About Emotion Editing

The paper introduces ROI‑ReFT, a method for emotional image editing that leverages neural representations from fMRI encoding models to guide visual changes while preserving content. By combining source visual responses with predicted affective changes, the approach achieves high target‑emotion accuracy (91.75%) and joint success (83.25%) on a curated set of 400 editing requests. The authors plan to release their code and dataset publicly.

By Xingzu Zhan, Lin Gu, Ruogu Fang
arXiv Computer Vision
Sep 3

AffectDelta: Beyond Emotion Labels for Image Editing

AffectDelta is a new image editing framework that moves beyond single emotion labels by modeling edits as transitions between eight‑dimensional emotion distributions. It uses a frozen Emotion Distribution Predictor to estimate the source state and a signed difference vector to encode the desired change, which is then translated into context‑dependent semantic and appearance modifications via a transition encoder and a diffusion backbone. The authors introduce AffectPair‑249K, a dataset of 248,841 source‑target pairs covering both cross‑category and within‑category transitions, and show that AffectDelta outperforms six baselines in affective alignment and content preservation.

By Xingzu Zhan, Lin Gu, Ruogu Fang
arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
Hugging Face Trending Papers
Sep 2

AffectDelta: Beyond Emotion Labels for Image Editing

AffectDelta is a new image editing framework that moves beyond single emotion labels by modeling edits as transitions between eight‑dimensional emotion distributions. It uses a frozen Emotion Distribution Predictor to estimate the source image’s affective state and encodes the signed difference to guide a diffusion backbone that applies context‑dependent semantic and appearance changes. The authors created a large AffectPair‑249K dataset of source‑target pairs and show that AffectDelta outperforms six baselines in both affective alignment and content preservation, with ablation studies supporting their design choices.

arXiv Computer Vision
Sep 7

RefDiT: Local Attribute Guidance in Reference-Based Image Generation

RefDiT is a new framework for reference-guided image generation that addresses the shortcomings of previous methods when handling complex scenes with multiple objects. It introduces local region guidance by decomposing a single identifier token into attribute-level signals, allowing the model to learn correspondences between tokens and specific regions of a reference image. The approach incorporates a low-rank adapter (LoRA) within a diffusion transformer (DiT) to adjust the inference prompt based on user-provided guidance context, thereby enabling more precise local attribute control.

By Rameshwar Mishra, Srikrishna Karanam, A V Subramanyam