← Back to all news
arXiv AI By K. P. Santoso, N. Z. Fadil, F. P. Harsanti, R. V. H. Ginardi, G. N. Iyer

Isotropic Yet Undecodable: The Sequential Content-Sufficiency Gap in Latent-Predictive Text Representations

Read the original on arXiv AI →

The paper investigates whether latent representations of text preserve the ordered information present in the input. Using an information‑theoretic decomposition, the authors show that even perfectly isotropic Gaussian representations can contain no target information, and that rank alone does not predict prediction risk. They introduce CANOPE, a non‑autoregressive framework with ordered latent canvases and geometric regularization, and demonstrate that while latent‑agreement and token‑grounded models achieve similar ranks, only the token‑grounded variant yields high positional recall and low word error rates on corrupted data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 17

Exact semantic readout from compressed vector representations

The paper investigates when compressed vector representations can provide exact linear or affine readouts for a finite lexicon’s truth conditions, establishing a necessary and sufficient row‑space condition. It shows that the augmented truth matrix’s rank determines the minimal dimension needed for exact linear (rank r) and affine (rank r − 1) readouts, and that exact readouts preserve Boolean connectives. Experiments on GloVe and word2vec embeddings reveal that while many predicates are linearly separable, none achieves exact affine recovery from pretrained embeddings, yet supervised transductive training can attain exact affine recovery at dimensions meeting the theoretical bound, preserving most of the original variance. "whyItMatters":"The results provide a precise mathematical criterion for when vector embeddings can faithfully encode logical truth conditions, informing both theoretical understanding and practical training of language models."

By Daniel Quigley
rag
More like this →
arXiv Computer Vision
Sep 2

Can Scene Text Recognition Read Rare Compositions?

The paper reports that scene text recognition models, while achieving 89–97% accuracy on standard benchmarks, perform significantly worse on rare word–trigram combinations, with a 10–18 point drop in accuracy at the rare‑word/rare‑trigram corner across multiple languages and models. Scaling the vision backbone improves overall accuracy but does not alleviate this corner‑specific deficit. The authors identify the autoregressive decoder’s lexical prior as the root cause and show that architectural changes—specifically moving from autoregressive to CTC decoding—yield the largest improvement for these rare compositions.

By Genpei Zhang
benchmarks
More like this →