arXiv:2608. 05995v1 Announce Type: new Abstract: Reliable uncertainty estimates are critical in safety-sensitive applications, where understanding the sources of predictive uncertainty is essential.
By Frieder Wizgall, Georg Tirpitz, Moritz Seiler, Kerstin Ritter, B\'alint Mucs\'anyi
arXiv:2511. 18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators.
By German Gritsai, Megan Richards, Maxime M\'eloux, Kyunghyun Cho, Maxime Peyrard
arXiv:2609.09245v1 Announce Type: new
Abstract: Repeated-sampling evaluations increasingly extrapolate pass@k far beyond the number n of samples collected per problem. We show that, in the pooled/ran...
By Pranav Singh, Prashant Singh
arXiv:2602. 15327v2 Announce Type: replace-cross Abstract: Machine learning model performance improvements tend to arise from competition and application.
By Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham Kakade
Repeated-sampling evaluations increasingly extrapolate pass@k far beyond the number n of samples collected per problem. We show that, in the pooled/random-task conditional-Binomial model, fixed-n succ...
The paper argues that relying solely on zero‑shot task accuracy is insufficient for evaluating quantized large language models (LLMs) because accuracy ignores changes in the full predictive distribution. It proposes a distribution‑sensitive framework that measures fidelity loss by computing statistical distances—such as Jensen‑Shannon Divergence and Total Variation Distance—between the full‑vocabulary output distributions of a full‑precision BF16 reference and its quantized counterparts. Experiments across five foundation architectures and four reasoning benchmarks show that these divergence metrics increase with stronger quantization, revealing distributional drift that top‑1 accuracy fails to capture, and suggest that mixed‑precision Q4_K schemes can offer lower divergence than uniform Q4_0 at comparable memory usage.
By Shahzeb Qamar, Lorenz Sparrenberg, Christian Bauckhage, Baha Rababah, Carson Leung, Murat Kantarcioglu, Cuneyt Gurcan Akcora, Rafet Sifa
The paper proposes a new approach to likelihood approximation for complex simulation models using a chain of aggregated normalizing flows. It describes a sequential estimation strategy where each set of bijective transformations is optimized independently, enabling scalable and controllable simulation routines for large‑scale data analysis. The framework also offers a surrogate model that facilitates sample generation, Bayesian inference, hypothesis testing, and uncertainty quantification.
By Getachew K Befekadu
The paper compares two main strategies for producing probabilistic forecasts of physical systems: generative models (e.g., diffusion or flow matching) and ensembles of deterministic models trained with the continuous ranked probability score (CRPS). It introduces a framework that evaluates both methods on diverse 2‑D spatiotemporal systems, measuring empirical coverage of predictive intervals, accuracy, and computational efficiency. The study finds that CRPS‑trained ensembles generally provide more reliable uncertainties and faster inference, while generative models trained in ambient space can match coverage but at higher latency. The authors also release AutoCast, a modular framework for both approaches, and AutoSim, a dataset generation tool.
By Sam F. Greenbury, Radka Jersakova, Paolo Conti, Marjan Famili, Christopher Iliffe Sprague, Edwin Brown, Farhan Feroz, Jason D. McEwen
arXiv:2609.24629v1 Announce Type: new
Abstract: A/B testing requires large sample sizes, long timelines, and significant costs. When auxiliary predictions of experimental outcomes are available from...
By Ziyad Benomar, Aymen Al Marjani, Paul Missault, Saab Mansour
arXiv:2501. 18897v4 Announce Type: replace-cross Abstract: Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification.
By Zijun Gao, Yan Sun, Han Su
arXiv:2509. 23385v5 Announce Type: replace-cross Abstract: Simulation-based inference (SBI) is transforming experimental sciences by enabling parameter estimation in complex non-linear models from simulated data.
By Pierre-Louis Ruhlmann, Michael Arbel, Florence Forbes, Pedro L. C. Rodrigues
arXiv:2602. 13362v2 Announce Type: replace-cross Abstract: A key challenge in probabilistic regression is ensuring that predictive distributions accurately reflect true empirical uncertainty.
By \'Ad\'am Jung, Domokos M. Kelen, Andr\'as A. Bencz\'ur