The paper introduces embedded Conditional Independence Tests (eCITs) to assess whether large language model (LLM) outputs contain additional information about an attribute beyond the source text. By embedding the source and output texts and applying existing CITs to these representations, the authors show that the null hypothesis can transfer from the original variables to their embeddings under sufficient conditions. Using a semi‑synthetic simulation design, they evaluate type I error control and power, and apply eCITs to German Parliament speeches, finding that LLM summaries reveal speaker faction and gender information beyond the original speech content.
By Marco Simnacher, Georg Keilbar, Benjamin K\"onig, Christoph Lippert, Sonja Greven
arXiv:2609.36307v1 Announce Type: new
Abstract: Partial Least Squares (PLS) regression extracts a few outcome-aligned directions in a high-dimensional X and is widely used across applied science, but...
By Pawe{\l} Lenartowicz, Hubert Plisiecki
arXiv:2511. 11041v2 Announce Type: replace-cross Abstract: We find that current sentence-embedding models produce outputs with a consistent bias: every embedding $e$ decomposes as $\tilde e + \mu$, where the mean $\mu$ is near-identical across all sentences.
By Xingyu Ren, Youran Sun, Haoyu Liang
The paper introduces Conditional Predictive Sufficient Statistics (CPSS) as a formal way to capture useful visual representations that preserve latent factors shared with future data while discarding noise. It shows that predicting the next image patch embedding with a cosine loss approximates maximum likelihood under a von Mises-Fisher model, and that stop‑gradient alone does not enforce sufficiency. Experiments on MNIST and CIFAR‑10 with small causal Transformers demonstrate that CPSS readouts outperform intermediate blocks, while removing stop‑gradient collapses embedding rank even when the pretext loss appears perfect.
By Yuzhou Hong
arXiv:2504. 12594v2 Announce Type: replace Abstract: Conditional independence testing is a critical component of feature screening, invariant statistical models, and causal discovery.
By Bijan Mazaheri, Jiaqi Zhang, Caroline Uhler
arXiv:2609.22337v1 Announce Type: cross
Abstract: Linear probing is the standard instrument for detecting social biases in the hidden representations of large language models. Yet reported probe accu...
By Mo Hai, Haifeng Li
arXiv:2512. 19510v2 Announce Type: replace Abstract: Conditional independence (CI) is central to causal inference, feature selection, and graphical modeling, yet it is untestable in many settings without additional assumptions.
By Alek Fr\"ohlich, Vladimir R. Kostic, Karim Lounici, Daniel Perazzo, Daniel Tiezzi, Massimiliano Pontil
The standard way to compare two text embeddings is cosine similarity. Scattered studies report that a different metric does better, but never pin down the geometric condition that decides when, or why.
The paper investigates the use of the squared norm of a whitened foundation‑model embedding as a training‑free likelihood surrogate. It shows that the apparent Gaussianity of whitened coordinates stems from the projection central limit theorem, not from a true joint Gaussian distribution, and that the norm is systematically over‑dispersed compared to a Gaussian reference. The authors explain that whitening reverses the encoder’s spectral hierarchy, concentrating norm contributions in near‑degenerate directions dominated by noise, and propose interpreting the squared norm as a Mahalanobis measure of semantic atypicality rather than a log‑likelihood.
By Mohammed Ahnouch, Lotfi Elaachak
arXiv:2607. 18305v1 Announce Type: cross Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text.
By Priyansh Srivastava, Romit Chatterjee
arXiv:2610.22655v1 Announce Type: cross
Abstract: The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space,...
By Nadir Kutluozen, Christian Intern\`o, David Klindt
arXiv:2606. 17426v1 Announce Type: cross Abstract: We consider the concentration properties of functions of infinitely exchangeable random variables.
By Fangyuan Lin, Spencer Frei, Victor H. de la Pena