arXiv AI August 28, 2026 By Yushe Cao, Dianxi Shi, Xing Fu, Xuechao Zou, Haikuo Peng, Xueqi Li, Chun Yu, Junliang Xing

Multivariate Diffusion Transformer with Decoupled Attention for High-Fidelity Mask-Text Collaborative Facial Generation

The paper introduces MDiTFace, a diffusion transformer designed for high‑fidelity mask‑text collaborative facial generation. It unifies tokenization of semantic masks and text, enabling synchronous multimodal feature interaction via stacked multivariate transformer blocks. A novel decoupled attention mechanism separates dynamic and static computations, allowing caching of static features and reducing mask‑condition overhead by over 94% while preserving performance, leading to superior facial fidelity and conditional consistency compared to existing methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 5

GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention

arXiv:2606. 06249v1 Announce Type: cross Abstract: Transformer-based multimodal models rely on attention mechanisms to integrate information across heterogeneous modalities.

By Giordano Cicchetti, Eleonora Grassucci, Danilo Comminiello

llms multimodal

arXiv Machine Learning

Aug 18

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens

arXiv:2506. 03096v2 Announce Type: replace-cross Abstract: Contrastive language-image pre-training aligns features of text-image pairs in a common latent space via distinct encoders for each modality.

By Christian Schlarmann, Francesco Croce, Nicolas Flammarion, Matthias Hein

llms rag diffusion multimodal

arXiv AI

Jun 12

HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

arXiv:2606. 13289v1 Announce Type: cross Abstract: Holistic visual tokenizers are fundamental to unified multimodal models (UMMs) as they map diverse visual inputs into a unified representation space.

By Guozhen Zhang, Xuerui Qiu, Yutao Cui, Tianhui Song, Changlin Li, Junzhe Li, Tao Huang, Xiao Zhang, Yang Li, Jianbing Wu, Miles Yang, Zhao Zhong, Liefeng Bo, Limin Wang

llms rag diffusion computer-vision nlp multimodal

arXiv AI

Jul 22

Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

arXiv:2607. 19344v1 Announce Type: cross Abstract: Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone.

By Rahul Sajnani, Yulia Gryaditskaya, Radom\'ir M\v{e}ch, Srinath Sridhar, Matheus Gadelha

llms diffusion multimodal

Hugging Face Trending Papers

Jul 23

GroupVideo: Multi-Identity Customized Text-to-Video Generation

Current identity customized video generation methodologies are predominantly limited to single-identity scenarios, as the lack of explicit identity separation mechanisms often leads to identity confusion in multi-identity settings. Existing multi-identity approaches, which directly extend single-identity frameworks by concatenating face images as input conditions, frequently result in unnatural facial expressions and motions, manifesting as the "copy-paste" phenomenon.

llms diffusion multimodal safety

arXiv Computer Vision

Sep 3

Reweighting Framewise Attention in Video Transformers for Facial Expression Understanding

The paper introduces MiRA, a plug‑in framework that reweights framewise attention in Vision Transformer video models to better capture subtle facial dynamics for expression recognition. MiRA computes frame‑level confidence and intra‑frame concentration from self‑attention maps, redistributing attention toward localized facial cues without adding trainable parameters. Two modes—an exact post‑softmax redistribution and a lightweight flashLite pre‑softmax approximation—are proposed, and experiments on facial expression recognition benchmarks show consistent gains over strong ViT baselines.

By Seongro Yoon, Donghyeon Cho, Jinsun Park, Fran\c{c}ois Br\'emond

llms computer-vision benchmarks