The paper introduces SCION, a single-model framework that jointly learns generative and self‑supervised representation tasks without relying on labels or pretrained encoders. SCION uses a pixel‑space encoder conditioned on a flow timestep and a shared embedding, training contrastive and flow‑matching objectives simultaneously with gradient‑norm balancing and stop‑gradient techniques. On ImageNet 256×256, SCION achieves state‑of‑the‑art FID scores (8.92 with JiT‑B and 5.89/3.47 with JiT‑L) surpassing prior class‑unconditional and pretrained‑conditioned methods.
arXiv:2609.36348v1 Announce Type: cross Abstract: Generative and representation learning remain asymmetrically connected: semantic representations are used to improve diffusion generation, whereas th...
arXiv:2610.19743v1 Announce Type: cross Abstract: Discriminative models are trained with task-specific objectives such as classification, detection, or representation learning, yet their practical ut...
arXiv:2608. 08309v1 Announce Type: cross Abstract: We argue that learning visual representations without labels requires a training signal jointly complete across three non-overlapping objectives: semantic invariance across augmented views, patch-level spatial prediction, and representational non-degeneracy.
arXiv:2605. 18324v2 Announce Type: replace-cross Abstract: Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders.
arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.
Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly.
ReVisIT is a train‑free framework that turns retrieved image‑label pairs into units of visual thought, combining structured class definitions, multimodal retrieval, and alternating user/assistant injection before joint decoding. On several benchmarks—including Fast Open MiniImageNet, Bongard‑OpenWorld, and the newly released MAAC‑Bench—ReVisIT achieves performance comparable to or surpassing large, trained models while using far fewer parameters. The approach demonstrates that high‑quality retrieval and a simple turns layer can provide a universal performance boost across diverse multimodal tasks.
The paper introduces Flow-Based Distribution Matching (FBDM), a non‑adversarial method that learns self‑supervised representations by aligning images to explicit geometric references through spherical conditional velocity regression. By using an ETF‑inspired reference, FBDM allows more reference components than the flow dimension while maintaining geometric separation, and it incorporates an alignment loss to bring augmented views closer together. Experiments on datasets from CIFAR to ImageNet show that FBDM performs nearly as well as adversarial distribution‑matching methods, achieves a 1.48‑ to 1.83‑fold speedup, and offers a theoretical bound on downstream misclassification rates.
The paper proposes a self‑supervised representation learning method that serves as a tokenizer for generative models. It supervises a representation loss by factorizing pointwise mutual information, estimated through differences between conditional predictions of a learned generative model. The authors demonstrate that this approach avoids dimensional collapse and feature suppression seen in methods like DINO and LeJEPA, scales predictably with compute, and achieves compute‑optimal performance over direct diffusion for pixel‑space generation, with strong scaling laws on ImageNet and GPIC.
The paper introduces Flow-Based Distribution Matching (FBDM), a non‑adversarial framework that learns self‑supervised representations using explicit geometric references and spherical conditional velocity regression. FBDM assigns augmented image views to shared target references while limiting reference usage, and employs an alignment loss to bring view representations closer. Experiments on datasets from CIFAR to ImageNet demonstrate that FBDM performs nearly as well as adversarial DM, outperforms existing SSL methods, and achieves a 1.48‑ to 1.83‑fold speedup with minimal GPU memory increase, while a theoretical analysis bounds downstream misclassification rates in terms of the pretraining loss.
arXiv:2602.05391v3 Announce Type: replace Abstract: Dataset distillation seeks to synthesize a compact surrogate dataset that enables performance comparable to training on the original dataset for do...
CoVA introduces a lightweight framework that composes VAE features online to provide stage‑wise guidance for flow‑matching training, improving semantic discrimination and spatial structure without extra inference overhead. By jointly training a small latent composer with class supervision, masked reconstruction, and flip consistency, CoVA aligns intermediate SiT features early and later uses detached composer posteriors for sample‑dependent soft‑label conditioning. On ImageNet 256×256, CoVA cuts SiT‑XL/2 FID from 17.2 to 7.4 in 400K steps and achieves an FID of 2.03 in only 400 epochs with classifier‑free guidance, adding just 8.5 M training‑only parameters.