Diffusion and generative media

Image, video and audio generation — diffusion models, flow matching and the systems built on top of them.

2,910 stories · RSS feed

arXiv Machine Learning
Jul 23

GaiaFlow: Semantic-Guided Diffusion Tuning for Carbon-Frugal Search

arXiv:2602. 15423v4 Announce Type: replace-cross Abstract: As the burgeoning power requirements of sophisticated neural architectures escalate, the information retrieval community has recognized ecological sustainability as a pivotal priority that necessitates a fundamental paradigm shift in model design.

By Rong Fu, Jia Yee Tan, Chunlei Meng, Shuo Yin, Xiaowen Ma, Wangyu Wu, Muge Qi, Simon Fong
arXiv Machine Learning
Jul 23

Geospatial Diffusion-based Evolution Synthesis (GeoDES) for Storm-Centered Weather Augmentation

arXiv:2607. 19522v1 Announce Type: new Abstract: While machine learning-based weather models hold significant promise, they struggle to predict the detailed structure of large-scale weather systems such as cyclonic storms.

By Sonia Cromp, Satya Sai Srinath Namburi GNVV, Youran Wang, Grace Kisslinger, Frederic Sala, James Booth, Allegra LeGrande
arXiv AI
Jul 23

StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation

arXiv:2607. 20174v1 Announce Type: cross Abstract: Existing human--object interaction (HOI) video generation methods are largely limited to offline short-video generation with complex driving conditions, making them unsuitable for real-time interactive applications.

By Zejing Rao, Haoxian Zhang, Xiaoqiang Liu, Yiping Meng, Guoxin Zhang, Pengfei Wan, Fan Tang, Tong-Yee Lee
arXiv AI
Jul 23

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

arXiv:2607. 20253v1 Announce Type: cross Abstract: In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes.

By Junyu Dai, Xinyue Fan, Weiqin Li, Xiangang Li, Yunjia Li, Bin Ma, Yukun Ma, Chongjia Ni, Yufei Shi, Haoxu Wang, Menglin Wu, Jianwei Yu, Huaicheng Zhang, Han Zhao, Shengkui Zhao, Haina Zhu
arXiv AI
Jul 23

Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting

arXiv:2607. 19404v1 Announce Type: cross Abstract: Multivariate time series encode structural patterns that unfold across multiple temporal scales, yet most forecasting backbones treat learned representations as transient byproducts of prediction, leaving the organizational geometry of these patterns underexploited.

By Xingsheng Chen, Deyu Yi, Siu-Ming Yiu
Hugging Face Trending Papers
Jul 22

Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering

In this report, we present a unified song generation framework capable of producing high-quality full-length music from lyrics, text descriptions, and musical attributes. The proposed framework supports three tasks: Lyrics-to-Song Generation, which generates complete songs from text descriptions, lyrics, and musical attributes; Instrumental Music Generation, which creates music without vocals; and Cover Song Generation, which reinterprets existing songs with different styles while preserving their melodic content.

Hugging Face Trending Papers
Jul 22

Antigen-specific Antibody Multi-modal Foundation Model for Functional Antibody Design

Antibodies are essential proteins that play a central role in immune recognition by binding specific antigen molecules. Although recent protein language models have enabled progress in single-chain protein modeling and generation, they often fall short in antigen-specific antibody design, where effective modeling requires explicit pairing between antibody and antigen, particularly at the epitope level.