arXiv Computer Vision

Long-Tail Adaptive Flow Matching with Explicit Conditional Consistency Guidance for Precise Multimodal Face Synthesis

The paper introduces EC²Face, a multimodal face synthesis framework that enhances semantic alignment by combining Explicit Conditional Consistency Guidance (ECCG) and Long‑Tail Adaptive Flow Matching (LAFM). ECCG enforces pixel‑level consistency between generated faces, textual descriptions, and semantic masks, while a temporal dynamic modulation adjusts supervision strength over diffusion timesteps. LAFM reweights spatial optimization signals according to attribute frequency, improving rare attribute synthesis without adding inference overhead. Experiments demonstrate that EC²Face outperforms baselines, achieving a 29.38% improvement in mask accuracy for rare attributes.

Hugging Face Trending Papers

Jul 7

FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration

Video face restoration (VFR) aims to recover high-quality and temporally consistent facial details from severely degraded video sequences; however, existing methods still struggle to balance spatial fidelity and temporal coherence under complex degradations. To address this, we propose FADRA, a frequency-aware diffusion framework with iterative residual adaptation specifically tailored for robust VFR.

diffusion fine-tuning benchmarks safety

arXiv AI

Sep 15

A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification

arXiv:2609.14419v1 Announce Type: cross Abstract: Person re-identification (ReID) is essential for multi-camera surveillance and tracking, yet remains difficult due to viewpoint and illumination chan...

By Leon Fernando, C Dombawala, P. Hettigoda, Vanodhya G. Warnasooriya, Ishara Neranjana, Rashmika Nawaratne

llms rag multimodal benchmarks

arXiv AI

Jul 31

Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model

arXiv:2607. 26742v1 Announce Type: cross Abstract: Zero-shot text-to-speech (TTS) clones a voice from a short audio prompt, but this reliance on reference audio is a barrier when only visual information is available, e.

By Carlos Mu\~noz-Romero, Jose A. Gonzalez-Lopez

diffusion multimodal

arXiv AI

Jul 21

Identity-Consistent Expression Fields: A Disentangled Neural Radiance Field Framework for Few-Shot Facial Expression Synthesis

arXiv:2607. 16287v1 Announce Type: cross Abstract: Neural Radiance Fields (NeRF) have enabled photorealistic novel-view synthesis of 3D scenes and, in the facial domain, have been extended to reconstruct and animate 3D face models from a small number of images.

By Minh Tran

arXiv Computer Vision

Sep 1

Identity-Conditioned Latent Consistency Distillation for Face Synthesis

arXiv:2608.31053v1 Announce Type: new Abstract: Diffusion models have achieved strong results in high-fidelity image synthesis, but their iterative sampling process makes large-scale generation compu...

By Tiago Kienen Chaves, Bernardo Biesseck, David Menotti

rag diffusion efficiency

Hugging Face Trending Papers

Jun 23

TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration

Face Video Restoration (FVR) aims to recover high-fidelity facial videos from degraded input while preserving identity and semantic consistency across frames. Existing methods often struggle to simultaneously address three key challenges: identity shift, viewpoint-entangled guidance, and perceptual realism.

rag diffusion benchmarks

arXiv AI

Jul 16

Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

arXiv:2508. 12466v2 Announce Type: replace-cross Abstract: Traditional multimodal learning approaches rely on alignment pre-training to bridge vision and language modalities, typically by projecting visual features into discrete text token spaces using large-scale image--text data.

By Xuhui Zhan, Tyler Derr

llms rag multimodal benchmarks safety

arXiv Computer Vision

Aug 26

LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter

arXiv:2505.23462v2 Announce Type: replace Abstract: Blind face restoration from low-quality images is a challenging task that requires not only high-fidelity image reconstruction, but also preservati...

By Runyi Li, Bin Chen, Jian Zhang, Radu Timofte

rag diffusion fine-tuning safety

arXiv AI

Jul 10

Reinforcing the Generation Order of Multimodal Masked Diffusion Models

arXiv:2607. 08056v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) have recently achieved substantial progress in natural language generation tasks.

By Yidong Ouyang, Zhe Wang, Sourav Bhabesh, Dmitriy Bespalov

llms diffusion multimodal benchmarks safety

arXiv Computer Vision

Sep 23

Vorch-Human: Unified Multi-Task Human-Centric Generation via Long-Horizon Continuation

Vorch-Human is a unified framework for human‑centric audio‑visual generation that handles multiple tasks—animating a person from speech, jointly generating speech and video from a voice reference, and synthesizing a scene from paired appearance and voice references—using a single dual‑stream audio‑video diffusion transformer. The model incorporates clean condition‑audio and condition‑video tokens, per‑token task embeddings, temporal position types, condition masks, and a shared multimodal prompt encoder to express diverse inputs such as driving speech, timbre examples, first frames, and subject images. A two‑level data pipeline supplies the necessary supervision by extracting speech, appearance, and timbre annotations from clips and linking consistent identity and outfit references across videos, while a frozen‑prefix recurrence enables long‑form audio‑driven generation with reduced boundary discontinuity and identity drift.

By Yang Ding, Haoran Yu, Xin Ma, Yulei Lu, Menglin Han, Yaole Wang, Siqian Yang, Gang Yue, Kaihao Zhang, Yaohui Wang, Lin Ma

llms rag diffusion multimodal data-engineering

arXiv AI

Jun 8

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.

By Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, Chonghan Liu, Hanzhen Zhao, Chen Liu, Xiaoxing Hu, Ziyue Qiao, Hao Tang, Xiaobin Hu, Chengwei Qin, Hui Xiong, Yu Qiao, Shuicheng Yan

llms rag fine-tuning multimodal safety

arXiv Computer Vision

Sep 14

Unified Text-Image Generation with Weakness-Targeted Post-Training

The paper introduces a post‑training approach that enables a single inference process to transition from text reasoning to image synthesis, eliminating the need for explicit modality switching. Using the 14B BAGEL model, the authors demonstrate that targeted post‑training data and reward‑weighted training improve multimodal image generation across four independent T2I benchmarks. The study highlights the benefits of joint text‑image generation and strategic data selection for enhancing T2I performance.

By Jiahui Chen, Philippe Hansen-Estruch, Xiaochuang Han, Yushi Hu, Emily Dinan, Amita Kamath, Michal Drozdzal, Reyhane Askari-Hemmat, Luke Zettlemoyer, Marjan Ghazvininejad

llms diffusion multimodal benchmarks

Related stories