The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully con...
arXiv:2609. 27860v1 Announce Type: new Abstract: A pointwise-unbiased one-bit compressor reconstructs every real input in expectation while transmitting one bit.
By Tao Jiang, Minbo Gao, Shaowei Cai
arXiv:2610.27402v1 Announce Type: cross
Abstract: Motivated by the notion of features from machine learning, we study the complexity of representing data that allows for evaluation with a given famil...
By Lunjia Hu, Anish Jayant, Peter Li, Princewill Okoroafor, Abhishek Shetty
arXiv:2604. 10727v2 Announce Type: replace-cross Abstract: Classical information-theoretic learning bounds typically rely on KL mutual information and moment-generating-function (MGF) arguments, which are well matched to bounded or sub-Gaussian losses but can be ineffective when losses or rewards are heavy-tailed.
By Huiming Zhang, Binghan Li, Wan Tian, Qiang Sun
arXiv:2608. 08416v1 Announce Type: new Abstract: Probably Approximately Correct (PAC) learning [Val84] is a fundamental learning model that has been extensively investigated.
By Steve Hanneke, Hongao Wang, Mingyue Xu
The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.
By Kai Niu, Ping Zhang
We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distribution over $\mathcal{X} \times \{0,1\}$, as in clas...
arXiv:2610.34703v1 Announce Type: new
Abstract: Simulation-based inference often encodes observations $Y$ into a fixed-dimensional summary $S$, with the posterior conditioning only on $S$. While exac...
By Vansh Bansal, Pradip Sharma Poudel, James G. Scott
arXiv:2606. 00703v1 Announce Type: cross Abstract: Low-precision pretraining (FP8, MXFP4, NVFP4) is now standard for frontier language models, yet the literature is almost entirely achievability -- algorithms and empirical scaling laws -- with no matching characterization of what is information-theoretically possible.
By Munsik Kim
arXiv:2610. 09206v1 Announce Type: new Abstract: Loss-difference conditional mutual information (ld-CMI) uses the smallest of the standard observations in the supersample hierarchy of generalization bounds: it measures what a learner's loss differences reveal about which candidate of each pair it was trained on.
By Hazar Yueksel
arXiv:2609.24260v1 Announce Type: cross
Abstract: We study the problem of \emph{adversarially robust} PAC learning. In this framework, the learner observes independent samples from an unknown distrib...
By Steve Hanneke, Amirreza Shaeiri
arXiv:2610. 12211v1 Announce Type: new Abstract: A verifier that accepts or rejects whole answers reveals little: under a flat prior over $k$-bit answers, zero error needs $2^k-1$ verifications.
By Hazar Yueksel