arXiv Computer Vision By Hongyang Du, Yunfei Xie, Junjie Ye, Jiawei Yang, Xiaoyan Cong, Haodong Zhang, Yongchao Huang, Haiyu Wu, Zongxia Li, Shihang Gui, Dawei Liu, Runhao Li, Jingcheng Ni, Chen Wei, Randall Balestriero, Yue Wang

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders

Read the original on arXiv Computer Vision →

The Flow has not summarised this story yet — read it at arXiv Computer Vision.

arXiv Machine Learning
Oct 12

Decoding Register Tokens Preserves Global Scene Details

The paper investigates whether keeping non‑spatial register tokens in vision transformer decoders improves image reconstruction and generation. Using representation autoencoders with frozen register‑equipped encoders, the authors show that joint self‑attention over both patch and register tokens yields better reconstruction quality on ImageNet‑1K, with lower FID scores and higher PSNRs compared to patch‑only baselines. For generation, a multi‑head attention pooling predictor recovers register features from sampled patches, further reducing FID by nearly 10% over the patch‑only approach.

By David Jacob Drexlin, Julius Hense, Jonas Dippel
arXiv Computer Vision
Oct 28

WithinVAE: Compact Visual Autoencoding with Self-Distilled Semantics

WithinVAE is a compact variational visual autoencoder that learns semantic structure internally using a sequential bottleneck and self‑distillation from an exponential‑moving‑average copy of itself. It achieves a four‑fold compression ratio on ImageNet‑1K, improving reconstruction metrics such as PSNR by 1.36 dB over RPiAE while using half the latent capacity. The model also boosts linear probing performance, demonstrating that semantic gains arise from self‑distillation rather than external supervision.

By Yuzhou Ji, Jiepeng Wang, Tianyu Jin, Lizhuang Ma, Haibin Huang, Chi Zhang, Xuelong Li