INCONVENIENT is a new benchmark that evaluates how machine‑learning models, especially large language models, generalize when trained on convenience samples that bias feature–target relationships. Unlike prior distribution‑shift tests that rely on natural domain splits, INCONVENIENT explicitly perturbs these relationships while keeping the deployment population unchanged, allowing controlled study of shift severity and direction. The benchmark covers 200 tabular classification and regression datasets and produces about 15,500 shifted training distributions, revealing significant performance degradation.
By Sara Rezaeimanesh, Chad Klochko, Kundan Thind, Mohammad Ghassemi
arXiv:2606. 09856v1 Announce Type: cross Abstract: Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable.
By Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake, Thomas L. Griffiths
The paper investigates how Large Language Models can be used to approximate domain expert priors for Bayesian Networks by extracting probabilistic knowledge about real‑world events. Experiments on eighty publicly available networks across domains such as healthcare and finance show that LLM‑derived conditional probabilities outperform random, uniform, and next‑token baselines. The authors also demonstrate that these LLM‑generated priors can refine data‑driven distributions, especially when data is scarce, and provide the first comprehensive baseline for evaluating LLM performance in probabilistic knowledge extraction.
By Aliakbar Nafar, Kristen Brent Venable, Zijun Cui, Parisa Kordjamshidi
arXiv:2607. 22961v1 Announce Type: new Abstract: Verbalized Machine Learning (VML) parameterizes a model as a natural-language prompt that an LLM evaluates as f(x; theta).
By Yan Zhang, Shikan Lian, Shibo Li
arXiv:2609.37891v1 Announce Type: cross
Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...
By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko
arXiv:2510. 19698v3 Announce Type: replace Abstract: Large Language Models (LLMs) can propose rules in natural language, sidestepping the need for a predefined predicate space in traditional rule learning.
By Yang Yang, Hua XU, Zhangyi Hu, Yutao Yue
The Marformer is a Transformer model designed to predict conditional marginal distributions of missing variables given observed data, enabling computation of Bayes risk and Value of Information. It learns a hidden-vector representation for each variable’s distribution and refines these through attention, similar to BERT’s masked language modeling but for probabilistic inference. Evaluations on synthetic Bayesian networks, discretized Gaussians, structured annotation data, and a real annotation dataset show that the Marformer matches or surpasses classical missing-data methods while being faster and not requiring explicit joint distribution modeling.
By Prabhav Singh, Xiheng Tom Wang, Haojun Shi, Jason Eisner
arXiv:2509.24988v2 Announce Type: replace-cross
Abstract: Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remain...
By Hanqi Xiao, Vaidehi Patil, Hyunji Lee, Elias Stengel-Eskin, Mohit Bansal
arXiv:2604. 16197v2 Announce Type: replace Abstract: Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability challenges on LLMs.
By Yide Ran, Jianwen Xie, Minghui Wang, Wenjin Zheng, Denghui Zhang, Chuan Li, Zhaozhuo Xu
arXiv:2607. 03882v1 Announce Type: cross Abstract: LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language.
By Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil
arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.
By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang
arXiv:2608. 11210v1 Announce Type: new Abstract: Bayesian calibration of process-based models requires a prior distribution for each model parameter.
By Patrik P. S\"uli, Gy\"orgy Eigner, Roland Holl\'os