arXiv Computer Vision
JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Oct 1
Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces
arXiv:2609.40362v1 Announce Type: new Abstract: We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models either model both language and q...
Hugging Face Trending Papers
Aug 31
NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference
Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision encoders with causal language models. Visual document...
arXiv AI
Sep 3
NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference
NeoMME is a family of 260M and 800M‑parameter multimodal‑native multilingual encoders that process text and raw image patches in a single bidirectional Transformer. Trained from scratch with a masked discrete‑diffusion objective conditioned on visible image patches, NeoMME supports a 16,384‑token context, enabling encoding of up to two 4K UHD images. In downstream tests, NeoMME‑Retriever models outperform all sub‑800M‑parameter baselines on the ViDoRe v3 benchmark and achieve twice the throughput of ColModernVBERT on an NVIDIA L40S, while hierarchical token pooling and asymmetric quantization compress embeddings 255× with minimal loss in retrieval performance.
arXiv AI
Oct 9
From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
The paper introduces SCION, a single-model framework that jointly learns generative and self‑supervised representation tasks without relying on labels or pretrained encoders. SCION uses a pixel‑space encoder conditioned on a flow timestep and a shared embedding, training contrastive and flow‑matching objectives simultaneously with gradient‑norm balancing and stop‑gradient techniques. On ImageNet 256×256, SCION achieves state‑of‑the‑art FID scores (8.92 with JiT‑B and 5.89/3.47 with JiT‑L) surpassing prior class‑unconditional and pretrained‑conditioned methods.
Hugging Face Trending Papers
Sep 15
FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Traditional multimodal representation learning and generation are two stages: a contrastive or self-supervised visual encoder is trained first, followed by a separate downstream generative model. This...
arXiv AI
Aug 19
Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models
The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.