arXiv Statistics ML
Intrinsic Information Rates of Summaries in Simulation-Based Inference
Read the original on arXiv Statistics ML →The Flow has not summarised this story yet — read it at arXiv Statistics ML.
Related stories
Hugging Face Trending Papers
6d ago
What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control
The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully con...
arXiv AI
2d ago
On the Fundamental Limits of Prefill-Based Decoding Length Prediction
The paper investigates the limits of predicting how many tokens a large language model will generate using only prefill-phase hidden states. It shows that current benchmarks mix predictor quality with task difficulty, leading to unstable accuracy when models, datasets, or sampling settings change. By modeling decoding as a stopped stochastic process, the authors derive bounds and an affine variance law that explain the observed normality of per‑prompt decode lengths and confirm that system error dominates model error.
arXiv AI
Jun 16
MA-SBI: Misspecification-Aware Simulation-Based Inference via Side-Channel Guidance
arXiv:2606. 16923v1 Announce Type: new Abstract: Simulation-based inference (SBI) of latent parameters is often hindered by simulator misspecification, the mismatch between simulated and real-world observations caused by inherent modeling simplifications.
arXiv Machine Learning
Oct 12
Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes
The paper introduces a two-way guidance framework that jointly learns summary statistics and posterior distributions for Bayesian simulation-based inference. By imposing a structural restriction on the learned summary, the method ensures that the posterior remains sufficient, while the posterior objective simultaneously steers the summary toward sufficiency. The authors provide theoretical guarantees on posterior estimation error and the deviation of the learned summary from sufficiency, outlining conditions on simulation budgets needed to recover increasingly concentrated posteriors as observation size grows.
arXiv AI
Jun 12
The Query Channel: Information-Theoretic Limits of Masking-Based Explanations
arXiv:2604. 16689v2 Announce Type: replace Abstract: Masking-based post-hoc explanation methods, such as KernelSHAP and LIME, estimate local feature importance by querying a black-box model under randomized perturbations.
arXiv Machine Learning
Oct 5
Metropolis-Hastings Dominates Importance Resampling for Policy Composition
arXiv:2610.03480v1 Announce Type: new Abstract: Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive...