Diffusion and generative media

Image, video and audio generation — diffusion models, flow matching and the systems built on top of them.

2,813 stories · RSS feed

arXiv AI

Jul 29

Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering

arXiv:2607. 25479v1 Announce Type: cross Abstract: Vision--Language Models (VLMs) are increasingly deployed through a model supply chain in which pretrained checkpoints, architecture definitions, text encoders, and exported computation graphs are distributed by third parties and reused across downstream services.

By Maria Rosaria Briglia, Igor Maljkovic, Antonio Emanuele Cin\`a, Luca Oneto, Iacopo Masi, Fabio Roli

llms rag diffusion nlp fine-tuning multimodal safety

arXiv AI

Jul 29

InDex: Empowering VLA Models with Intent-Conditioned Arm-Hand Coordination for Dexterous Manipulation

arXiv:2606. 12109v2 Announce Type: replace-cross Abstract: Pre-trained Vision-Language-Action (VLA) models provide useful semantic and spatial priors, yet their parallel-gripper action interfaces do not specify how those priors should be realized by a dexterous hand.

By Chuanke Pang, Junyi Huang, Zhijun Zhao, Yaobing Wang, Kun Xu, Xilun Ding

diffusion robotics multimodal

arXiv AI

Jul 29

CIFNet: An Analytic Neural Learning Framework for Efficient and Calibrated Class-Incremental Learning

arXiv:2509. 11285v2 Announce Type: replace-cross Abstract: Class-Incremental Learning (CIL) in deep neural networks is conventionally framed as an iterative gradient-based optimization problem, incurring high computational cost, hyperparameter sensitivity, and risk of catastrophic forgetting.

By Alejandro Dopico-Castro, Oscar Fontenla-Romero, Bertha Guijarro-Berdi\~nas, Amparo Alonso-Betanzos

rag diffusion safety

arXiv AI

Jul 29

$\pi\mathbf{R}^2$: Reactive Real-time Flow Policies

arXiv:2607. 26055v1 Announce Type: cross Abstract: Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones.

By Sungjae Park, Shubham Tulsiani

rag diffusion robotics multimodal

arXiv AI

Jul 29

MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities

arXiv:2607. 25948v1 Announce Type: cross Abstract: Any-to-any models predict any modality from any combination of others within a single network, a formulation used in multimodal vision and vision-language models, and increasingly in scientific domains such as ecology and astronomy.

By Mingqiao Ye, Zhaochong An, Zhitong Gao, Xian Liu, Fran\c{c}ois Fleuret, Chuan Li, Amir Zadeh, Serge Belongie, Afshin Dehghan, Jesse Allardice, David Mizrahi, O\u{g}uzhan Fatih Kar, Roman Bachmann, Amir Zamir

llms diffusion multimodal benchmarks

arXiv AI

Jul 29

OmniPhys: Knowledge-Graph-Driven Benchmarking and Collective Optimization for Physical Commonsense in Text-to-Image Generation

arXiv:2607. 25641v1 Announce Type: cross Abstract: While text-to-image models exhibit remarkable visual fidelity, they frequently violate fundamental physical commonsense.

By Yajing Xu, Yarong Lan, Jiaoyan Chen, Yichi Zhang, Jeff Z. Pan, Mingchen Tu, Zhizhen Liu, Wen Zhang, Huajun Chen

diffusion benchmarks safety

arXiv AI

Jul 29

OrganLens: Organ-Specific Representation Learning for CT Foundation Models

arXiv:2607. 25164v1 Announce Type: cross Abstract: A CT examination captures multiple organs, but many biomedical questions concern abnormalities, prognosis, or longitudinal change in a specific organ.

By Zhixuan Ge, Anqi Li, Sadeer Al-Kindi, Hanwen Xu, Wei Qiu

diffusion computer-vision efficiency

arXiv AI

Jul 28

Controlling Embedding Spaces with Text-Conditioned Transformations

arXiv:2607. 22919v1 Announce Type: cross Abstract: Multimodal embedding spaces in models like CLIP enable powerful capabilities such as semantic similarity retrieval and cross-modal zero-shot classification.

By Joseph Fioresi, Fabian Caba Heilbron, Pankaj Nathani, Mubarak Shah, Kushal Kafle

rag diffusion multimodal benchmarks

arXiv Machine Learning

Jul 28

The balance between compactness and forecast accuracy of data-driven latent-space reduced-order models in controlled wake flows

arXiv:2607. 24569v1 Announce Type: cross Abstract: Model-based active flow control requires predictive models that are accurate, stable, and fast enough for real-time optimisation.

By Alberto Solera-Rico, Patricia Garc\'ia-Caspue\~nas, Carlos Sanmiguel Vila, Stefano Discetti

diffusion reinforcement-learning

arXiv AI

Jul 28

ChangeFlow -- Latent Rectified Flow for Change Detection in Remote Sensing

arXiv:2605. 15375v2 Announce Type: replace-cross Abstract: Remote sensing change detection (RSCD) localises changes between two images of the same geographic region.

By Bla\v{z} Rolih, Matic Fu\v{c}ka, Filip Wolf, Luka \v{C}ehovin Zajc

diffusion benchmarks

arXiv Machine Learning

Jul 28

Breaking the Synthetic-Real Domain Shortcut for Training-Free Generative Replay-based Class Incremental Learning

arXiv:2607. 22994v1 Announce Type: cross Abstract: Class-incremental learning (CIL) requires models to continuously acquire new knowledge while avoiding catastrophic forgetting.

By Tao Zhang, Qixuan Fan, Yiyuan Liang, Yanjie Wang, Song Yan, Tian Tian, Jiahuan Zhou, Luxin Yan, Sheng Zhong, Xu Zou

diffusion benchmarks safety

arXiv Machine Learning

Jul 28

Catalyst Diffusion Transformer: Generative Inverse Design of Heterogeneous Catalysts

arXiv:2607. 24272v1 Announce Type: cross Abstract: The vast chemical design space and complex, interdependent design variables make catalyst discovery for targeted properties highly labor- and resource-intensive.

By Hayoung Doo, Dong Hyeon Mok, Seoin Back, Jonggeol Na

llms diffusion

arXiv AI

Jul 28

LU-500: A Logo Benchmark for Concept Unlearning

arXiv:2607. 24101v1 Announce Type: cross Abstract: Concept unlearning is increasingly used to limit the reproduction of protected or unsafe visual concepts in text-to-image models.

By Keyu Li, Jin Gao, Jialing Zhang, Dequan Wang

agents diffusion fine-tuning benchmarks

arXiv Machine Learning

Jul 28

MMOE: Modernizing Diffusion Transformers with Efficient Expert Design

arXiv:2607. 24665v1 Announce Type: cross Abstract: Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.

By Yanhao Jia, Jiepeng Wang, Haibin Huang, Chi Zhang, Erik Cambria, Xuelong Li

llms diffusion efficiency

arXiv AI

Jul 28

Spectral Dynamics of Semantic Drift in Clinical Multi-Agent Language Model Networks

arXiv:2607. 22758v1 Announce Type: cross Abstract: The integration of iterative LLMs within multi-agent diagnostic frameworks requires a rigorous quantitative reevaluation of underlying communication topologies.

By Amritesh Banerjee

llms rag agents diffusion safety

arXiv Machine Learning

Jul 28

FRIGID: Scaling Diffusion-Based Molecular Generation from Mass Spectra at Training and Inference Time

arXiv:2604. 16648v2 Announce Type: replace Abstract: Tandem mass spectrometry is prominent in scientific discovery workflows for identifying unknown small molecules, yet high-throughput structural elucidation remains challenging.

By Montgomery Bohde, Hongxuan Liu, Mrunali Manjrekar, Magdalena Lederbauer, Shuiwang Ji, Runzhong Wang, Connor W. Coley

llms diffusion efficiency benchmarks

arXiv AI

Jul 28

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

arXiv:2607. 24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw prompts from web sources or LLM templates and score them with untrained, generic multimodal models.

By Shengyi Wang, Niantong Li, Guangzheng Hu, Hong Qi, Fei Ding, Weixu Qiao, Jinlin Wang, Xiaotong Lv, Peng Han, Zimeng Li, Fanshu Ding, Yushu Wang, Han Wu, Jingjing Chen, Chongxiao Wang, Yanhao Wu, Chenglong Huang, Xiaoqian Zhu, Jie Tian, Hua Li, Jingjing Fan, Mingshuang Tang, Zhong Li, Hengxia Qiang, Weibin Chen, Jinyang Zhen, Bing Zhao, Lin Qu, Jing Li, Hu Wei

llms agents diffusion multimodal benchmarks safety

arXiv AI

Jul 28

Escaping the Euclidean Void: Manifold-Informed Flow Matching for Sequential Recommendation

arXiv:2607. 23762v1 Announce Type: cross Abstract: Conventional recommenders capture users' preferences by optimizing observed user-item relations, whereas continuous generative recommendation additionally learns the trajectory of synthesizing a target item.

By Dengzhao Fang, Jingtong Gao, Yu Li, Xiangyu Zhao, Yi Chang

rag diffusion benchmarks

arXiv Machine Learning

Jul 28

To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansion

arXiv:2607. 23492v1 Announce Type: cross Abstract: Concept erasure techniques (CETs) edit text-to-image diffusion models to erase undesired targets such as NSFW content or copyrighted styles, while preserving model utility on benign concepts.

By Shaswati Saha, Rajasekhar Anguluri, Manas Gaur

llms diffusion multimodal

arXiv AI

Jul 28

Fashion-3DLR: A Controllable 3D Garment Generation Using Pairwise Fashion Elements for Intelligent Design

arXiv:2607. 23189v1 Announce Type: cross Abstract: AI-generated content (AIGC) has made significant progress, with 2D generative models becoming ready-to-use tools for the digital fashion industry.

By Shenghao Yang, Hongtao Zhang, Yuhan Yi, Zhihao Tang, Zihao Cui, Lian Wen, Han Yan, Yuan Gao, Mingbo Zhao

llms diffusion benchmarks