From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model

The paper introduces SCION, a single-model framework that jointly learns generative and self‑supervised representation tasks without relying on labels or pretrained encoders. SCION uses a pixel‑space encoder conditioned on a flow timestep and a shared embedding, training contrastive and flow‑matching objectives simultaneously with gradient‑norm balancing and stop‑gradient techniques. On ImageNet 256×256, SCION achieves state‑of‑the‑art FID scores (8.92 with JiT‑B and 5.89/3.47 with JiT‑L) surpassing prior class‑unconditional and pretrained‑conditioned methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• rag
• diffusion