Hugging Face Blog

Remote VAEs for decoding with Inference Endpoints ๐Ÿค—

arXiv AI
Jul 3

Low-Latency Task-Oriented Image Transmission with Opportunistic Spectrum Access

arXiv:2607. 01921v1 Announce Type: cross Abstract: Communication systems designed for reliable data reconstruction, rather than task-oriented communication, typically rely on separate source and channel coding and incur high latency under limited spectrum availability and fading channels.

By Jo\~ao Henrique Inacio de Souza, Mattia Merluzzi, Mateus P. Mota, Beatriz Soret, Petar Popovski
arXiv Machine Learning
1d ago

WhiteMatter: All-to-All Cross-Layer Connections via KV Mixing

WhiteMatter introduces a novel architecture for Transformers that connects every attention layer to representations from all layers of each past token, allowing connection weights to vary across consumer layers and adapt to the source token. The design uses a router to mix the $L$ layer states of each token into $k$ KV channels, which are cached for subsequent tokens; each consumer layer attends to one channel. Experiments show that WhiteMatter outperforms a vanilla Transformer with 50% more layers and maintains most of this advantage even when the KV-cache is compressed by 50%.

By Wenbo Zhang, Xiang Ren