arXiv Machine Learning June 24, 2026 By Leon Chlon, Fatima Sheaib, Zein Khamis, Maggie Chlon, Mahdi El Zein, MarcAntonio M. Awada

LLMs are Bayesian, In Expectation, Not in Realization

Read the original on arXiv Machine Learning →

arXiv:2507. 11768v3 Announce Type: replace-cross Abstract: Bayesian accounts of in-context learning face a direct objection: exact posterior predictives for exchangeable data are invariant to task-preserving order, yet transformers change next-token probabilities when the same examples are serialized differently.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 8

Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication

arXiv:2509. 11208v3 Announce Type: replace-cross Abstract: Transformers used for evidence-grounded binary adjudication (e.

By Leon Chlon, Ahmed Karim, Maggie Chlon, MarcAntonio Awada

llms safety

arXiv Machine Learning

Jul 22

Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

arXiv:2607. 18804v1 Announce Type: new Abstract: In the \emph{latent posterior model} of transformer behavior, the next-token distribution arises from a posterior over latent predictive models conditioned on the context, mixed to generate continuations.

By Garrett Baker, Vinayak Pathak, Daniel Murfet, Susan Wei

llms

arXiv Machine Learning

Jul 21

What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach

arXiv:2607. 17060v1 Announce Type: new Abstract: A Bayes-filtered transformer (BFT) is a transformer trained on sequences that are generated in two steps: first a latent task is drawn from a prior, then observations are drawn conditional on that task.

By Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan Wei

llms diffusion safety

arXiv Machine Learning

Jun 18

Decomposing Prediction Mechanisms for In-Context Recall

arXiv:2507. 01414v2 Announce Type: replace Abstract: We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall.

By Sultan Daniels, Dylan Davis, Dhruv Gautam, Wentinn Liao, Gireeja Ranade, Anant Sahai

llms efficiency

arXiv Machine Learning

Jul 30

From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models

arXiv:2607. 26504v1 Announce Type: new Abstract: Many discrete reasoning tasks, such as code generation, are inherently non-causal: programmers move between high-level structure and local details, a process we call any-order inference.

By Seunggeun Kim, Jaeyeon Kim, Taekyun Lee, Yuyuan Chen, Yilun Du, Sham Kakade, Sitan Chen

llms diffusion

arXiv Machine Learning

Jul 22

PAC--Bayes Bounds on Quotient Parameter Spaces: Geometry-induced Implicit-Bias Priors

arXiv:2607. 18422v1 Announce Type: new Abstract: Overparameterized models often have continuous parameter symmetries, so different parameters define the same predictor.

By Nicola Aladrah, Fabio Anselmi

safety

Related stories

Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication

Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach

Decomposing Prediction Mechanisms for In-Context Recall

From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models

PAC--Bayes Bounds on Quotient Parameter Spaces: Geometry-induced Implicit-Bias Priors