arXiv Machine Learning July 21, 2026 By Afiq Abdillah Effiezal Aswadi, Haotong Ma, Susan Wei

What does a Bayes-filtered transformer believe? A predictive Monte Carlo approach

Read the original on arXiv Machine Learning →

arXiv:2607. 17060v1 Announce Type: new Abstract: A Bayes-filtered transformer (BFT) is a transformer trained on sequences that are generated in two steps: first a latent task is drawn from a prior, then observations are drawn conditional on that task.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jul 22

Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

arXiv:2607. 18804v1 Announce Type: new Abstract: In the \emph{latent posterior model} of transformer behavior, the next-token distribution arises from a posterior over latent predictive models conditioned on the context, mixed to generate continuations.

By Garrett Baker, Vinayak Pathak, Daniel Murfet, Susan Wei

llms

arXiv Machine Learning

Jun 24

LLMs are Bayesian, In Expectation, Not in Realization

arXiv:2507. 11768v3 Announce Type: replace-cross Abstract: Bayesian accounts of in-context learning face a direct objection: exact posterior predictives for exchangeable data are invariant to task-preserving order, yet transformers change next-token probabilities when the same examples are serialized differently.

By Leon Chlon, Fatima Sheaib, Zein Khamis, Maggie Chlon, Mahdi El Zein, MarcAntonio M. Awada

llms benchmarks

arXiv AI

Jun 30

Transformer Architectures as Complete Bayes Processes: A Formal Proof in the Measure-Theoretic Kernel Framework

arXiv:2606. 30440v1 Announce Type: cross Abstract: We present a complete formal proof that transformer architectures, when their internal update mechanisms satisfy a Bayes joint-distribution condition, implement exact Bayesian posterior inference.

By Haobo Yang

llms

arXiv Machine Learning

Jul 21

Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference

arXiv:2602. 04596v2 Announce Type: replace-cross Abstract: Bayes-filtered transformers are transformers meta-learned on sequences from a prior predictive distribution to approximate the corresponding posterior predictive distribution.

By Sandra Fortini, Kenyon Ng, Sonia Petrone, Judith Rousseau, Susan Wei

llms benchmarks

arXiv Machine Learning

Jul 28

Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses

arXiv:2607. 22961v1 Announce Type: new Abstract: Verbalized Machine Learning (VML) parameterizes a model as a natural-language prompt that an LLM evaluates as f(x; theta).

By Yan Zhang, Shikan Lian, Shibo Li

llms benchmarks

arXiv Machine Learning

Jun 19

Multi-Task Bayesian In-Context Learning

arXiv:2606. 20538v1 Announce Type: new Abstract: Bayesian predictive inference provides a principled framework for uncertainty quantification, data efficiency, and robust generalization.

By Qingyang Zhu, Eric Karl Oermann, Kyunghyun Cho

llms benchmarks

Related stories

Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

LLMs are Bayesian, In Expectation, Not in Realization

Transformer Architectures as Complete Bayes Processes: A Formal Proof in the Measure-Theoretic Kernel Framework

Uncertainty Decomposition for Bayes-Filtered Transformers via Bayesian Predictive Inference

Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses

Multi-Task Bayesian In-Context Learning