arXiv Machine Learning

Knowledge as Orbit: Finite Collections as Phases of an Exactly Periodic Latent Generator

The paper proposes storing finite collections of knowledge as the orbit of a single compact latent generator that cycles exactly back to its starting point. By encoding each item as a phase of a fixed rotation in a learned latent space and decoding all phases with a shared network, the method guarantees exact closure through a discrete Fourier operator. Experiments on images and video clips show that this exactly periodic operator outperforms general learned or norm‑preserving operators, achieving comparable or better fidelity while eliminating visible seams and enabling efficient compression.

arXiv Computation and Language
Sep 4

Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs

The study investigates how long‑video language models decide which frames to keep, compress, and reuse, testing each decision in isolation across six selection rules, three benchmarks, and two answering models. It finds that selecting frames based on queries yields the biggest performance boost, that halving spatial resolution costs little, and that reallocating saved tokens to more compressed frames can further improve accuracy. The work also highlights the importance of a unified evaluation harness to avoid misleading comparisons.

By Prakhar Khatri
arXiv AI
Jul 3

Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability

arXiv:2607. 01799v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) decompose internal activations of neural networks into sparse linear combinations of learned features by fitting an overcomplete dictionary $\mathbf{W}\in\mathbb{R}^{m\times n}$ with $m<n$, and inferring a sparse code $\mathbf{x}\in\mathbb{R}^n$ from $\mathbf{h}\approx\mathbf{W}\mathbf{x}$.

By Rodrigo Mendoza-Smith
arXiv AI
Jul 1

Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readouts

arXiv:2606. 30705v1 Announce Type: cross Abstract: Deterministic few-step generation succeeds on continuous image latents but collapses to incoherent text on continuous text latents, and we show the cause is geometric rather than a training or scaling deficiency: a smooth, regularity-limited deterministic map cannot resolve a discrete branch choice before a sharp categorical readout, so few-step failure is governed by decoder sharpness, not transport accuracy.

By Zhongyao Wang