FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
arXiv:2609.37775v1 Announce Type: cross Abstract: Pretrained visual representations support image generation, but may not fully preserve the fine-grained details needed for faithful reconstruction. M...
The paper investigates whether keeping non‑spatial register tokens in vision transformer decoders improves image reconstruction and generation. Using representation autoencoders with frozen register‑equipped encoders, the authors show that joint self‑attention over both patch and register tokens yields better reconstruction quality on ImageNet‑1K, with lower FID scores and higher PSNRs compared to patch‑only baselines. For generation, a multi‑head attention pooling predictor recovers register features from sampled patches, further reducing FID by nearly 10% over the patch‑only approach.
arXiv:2610.09242v1 Announce Type: new Abstract: Representation Autoencoders (RAEs) generate images from pre-trained visual fea- tures, but their dense token grids make generative modeling expensive....
arXiv:2605. 18324v2 Announce Type: replace-cross Abstract: Representation Autoencoders (RAE) replace traditional VAE with pretrained vision encoders.
arXiv:2608. 01298v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks.
WithinVAE is a compact variational visual autoencoder that learns semantic structure internally using a sequential bottleneck and self‑distillation from an exponential‑moving‑average copy of itself. It achieves a four‑fold compression ratio on ImageNet‑1K, improving reconstruction metrics such as PSNR by 1.36 dB over RPiAE while using half the latent capacity. The model also boosts linear probing performance, demonstrating that semantic gains arise from self‑distillation rather than external supervision.