arXiv Machine Learning August 4, 2026 By Yichao Cai, Javen Qinfeng Shi

On the Identifiability of Masked Prediction: Mode Blindness and Mask Schedules

Read the original on arXiv Machine Learning →

arXiv:2608. 01383v1 Announce Type: new Abstract: Masked prediction learns representations by fitting a schedule-weighted family of conditional laws, but it remains unclear when near-optimal conditional prediction pins down the underlying joint law.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 22

The Information Shadow: Measuring Structural Limits on What Language Models Can Learn

arXiv:2607. 18305v1 Announce Type: cross Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text.

By Priyansh Srivastava, Romit Chatterjee

llms benchmarks safety

arXiv Machine Learning

Aug 12

Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability

arXiv:2608. 10172v1 Announce Type: new Abstract: Mechanistic interpretability explains models by identifying circuits inside them, but has no way to tell whether a circuit is a property of the model or an artifact of the method that found it.

By Ashim Dhor, Pin-Yu Chen

llms safety

arXiv Machine Learning

Jun 11

A prior-free blind detection of information leakage from model predictions

arXiv:2606. 11267v1 Announce Type: new Abstract: Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise.

By Laurence A. Jacobs

arXiv Machine Learning

Jun 4

Rethinking Incompleteness: Formalizing Protocol Divergence and Train-Once Learning for Robust IMVC

arXiv:2606. 04857v1 Announce Type: new Abstract: Standard IMVC evaluation retrains separate models for different missing-data configurations.

By Haolu Liu, Xiyue Wang, Xuanting Xie, Liangjian Wen, Zhao Kang

llms benchmarks

arXiv Machine Learning

Jun 8

Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws

arXiv:2606. 06888v1 Announce Type: new Abstract: Classical scaling laws for language model pretraining balance model size against training dataset size under a fixed compute budget, assuming abundant data and a single pass over the corpus.

By Zhiwei Xu, Shihao Wu, Hanseul Cho, Wei Hu, Yixin Wang

llms diffusion

arXiv Machine Learning

1d ago

Cross-Entropy Risk Estimation for Language Models: Inconsistency Must Be Dense, and the Holdout Method Is No Exception

arXiv:2608. 15798v1 Announce Type: new Abstract: Language models are compared by their held-out per-token cross-entropy risk---the quantity scaling laws are fitted to.

By Hanti Lin

llms

Related stories

The Information Shadow: Measuring Structural Limits on What Language Models Can Learn

Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability

A prior-free blind detection of information leakage from model predictions

Rethinking Incompleteness: Formalizing Protocol Divergence and Train-Once Learning for Robust IMVC

Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws

Cross-Entropy Risk Estimation for Language Models: Inconsistency Must Be Dense, and the Holdout Method Is No Exception