arXiv Computation and Language
Statistical Mechanics of Semantic Compression
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Jun 30
A Synonymous Variational Perspective on the Rate-Distortion-Perception Tradeoff
arXiv:2604. 14603v2 Announce Type: replace-cross Abstract: The fundamental limit of natural signal compression has traditionally been characterized by classical rate-distortion (RD) theory through the tradeoff between coding rate and reconstruction distortion, while the rate-distortion-perception (RDP) framework introduces a divergence-based measure of perceptual quality as a modeling principle, leaving its theoretical origin unclear.
arXiv Machine Learning
Jul 17
Gibbs randomness-compression proposition
arXiv:2505. 23869v4 Announce Type: replace-cross Abstract: A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a compression process.
arXiv AI
Jun 30
Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
arXiv:2606. 30625v1 Announce Type: cross Abstract: Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes.
arXiv Machine Learning
22h ago
Emergent phases of superposition: from partial to full representation
arXiv:2609.36455v1 Announce Type: new Abstract: Large language models are thought to represent features by vectors in a hidden space of dimension given by the model's width. Superposition, in which m...
arXiv Machine Learning
Jun 24
Similarity of Neural Network Representations in Superposition
arXiv:2604. 00208v2 Announce Type: replace Abstract: Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features.
Hugging Face Trending Papers
Sep 21
Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities
The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds only weak pointwise similarity between mo...