arXiv Machine Learning

Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics

arXiv:2606. 11833v1 Announce Type: new Abstract: Flow matching and diffusion models enable conditional generation across domains ranging from images to proteins, with recent extensions to out-of-distribution contexts.

Hugging Face Trending Papers
Jun 3

Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction

Understanding the relationship between deep visual representations and the human visual system is a fundamental challenge in computational neuroscience. While modern vision models achieve strong performance in image recognition, their correspondence with the hierarchical organization of the human visual cortex remains an open question.

arXiv AI
Jul 8

Few Channels Draw The Whole Picture: Revealing Massive Activations in Diffusion Transformers

arXiv:2605. 13974v2 Announce Type: replace-cross Abstract: Diffusion Transformers (DiTs) and related flow-based architectures are now among the strongest text-to-image generators, yet the internal mechanisms through which prompts shape image semantics remain poorly understood.

By Evelyn Turri, Davide Bucciarelli, Sara Sarto, Lorenzo Baraldi, Marcella Cornia
arXiv Machine Learning
Oct 12

Rethinking the Roles of Diffusion and Decoding: Predictive-State Flows for Parallel Language Generation

The paper proposes Predictive-State Flows (PSF), a new architecture for continuous diffusion language models that separates the diffusion prior’s role to generate high‑level predictive states and the decoder’s role to resolve token‑level choices. PSF partitions sequences into blocks, uses a rectified‑flow prior to produce predictive states, and then a block‑causal decoder generates all blocks in parallel, reducing token‑level sequential depth. Experiments on OpenWebText show that a 197M‑parameter PSF model with eight predictive states achieves a generative perplexity of 22.8 and a MAUVE@512 of 0.93 while using far fewer FLOPs than a comparable ELF‑B model.

By Jinjin He, Xinsong Feng, Lucheng Fu, Sinan Wang, Bo Zhu, Jiayun Wang
arXiv Computation and Language
Oct 12

Exploring Attention Sinks in Text-to-Image Transformers from Diffusion and Autoregressive Models

The paper investigates attention sinks—tokens or positions that attract disproportionate attention—in text-to-image transformers, comparing bidirectional diffusion models and causal autoregressive models. It finds that both model types exhibit attention sinks, but with distinct patterns: autoregressive models focus on the first and global tokens, while diffusion models develop sinks near modality boundaries, within text regions, and at spatial corners. Perturbation experiments show that sink-related heads influence generation more than non-sink heads, and introducing register tokens can redistribute attention without harming performance.

By Binglei Li, Mengping Yang, Zhiyu Tan, Wenxu Qian, Junping Zhang, Hao Li
arXiv AI
Aug 11

Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching

arXiv:2608. 08135v1 Announce Type: cross Abstract: Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches rather than whole volumes, and train a separate model for each translation task.

By Daniele Molino, Alessio Zoboli, Camillo Maria Caruso, Valerio Guarrasi, Paolo Soda