arXiv Machine Learning

Understanding Uncertainty Sampling via Equivalent Loss

arXiv Statistics ML
Nov 9

Reweighting for Transfer Classification: Exact Risk Conversion and Minimax Adaptation

The paper investigates how reweighting a source training mixture affects target classification risk. It derives exact excess hinge‑risk bounds for binary classification, provides a two‑sample oracle inequality for weighted SVMs, and establishes minimax rates that separate source sampling uncertainty from relation uncertainty. The results show that the usefulness of a source depends on the training problem it creates, the relevance of its accuracy to the target, and the ability to apply corrective weighting.

By Aoran Chen, Yang Feng
arXiv Machine Learning
Sep 10

Large Classification-Risk-Optional Label Acquisition

arXiv:2609.06873v1 Announce Type: cross Abstract: We study how a limited labeling budget should be allocated to minimize multiclass zero-one classification risk. We consider parametric classification...

By F. Setoudehtanzangi, Geoffrey J. McLachlan
arXiv Machine Learning
Sep 30

Learning Distributionally Robust First-Order Methods for Convex Optimization

The paper introduces a distributionally robust method for learning hyperparameters of first‑order convex optimization algorithms. By minimizing a Wasserstein‑robust performance estimation problem over a dataset of problem instances, the approach interpolates between classical learning‑to‑optimize (L2O) and worst‑case PEP design. The authors solve the resulting problem with stochastic gradient descent, provide high‑probability risk bounds, and demonstrate that the learned algorithms outperform both worst‑case optimal and vanilla L2O baselines on logistic regression, LASSO, and linear programming tasks.

By Vinit Ranjan, Jisun Park, Bartolomeo Stellato
arXiv Machine Learning
Oct 16

The Tilted Sampling Problem: Optimism, Posterior Sampling, and Logarithmic Regret

The paper introduces the tilted sampling problem, where an online learner aims to sample from a Gibbs distribution defined by an unknown reward function. It defines sampling regret as the temperature‑normalized cumulative KL divergence between the learner’s distribution and the target Gibbs distribution, linking it to entropy‑regularized reward maximization. The authors develop sampling analogues of UCB and Thompson sampling, proving that sampling regret can achieve logarithmic growth in time, unlike standard regret which scales as ×O(√T).

By Mohammad Pedramfar, Siamak Ravanbakhsh
arXiv Machine Learning
Sep 22

Classification with Abstention Under Class-Conditional Error Constraints

The paper investigates binary classification with abstention under separate class‑conditional error constraints, aiming to minimize abstention while keeping both error types below specified thresholds. It derives the distribution‑free minimax rate of excess abstention risk, introduces surrogate‑loss formulations for computational feasibility with models like neural networks, and provides finite‑sample guarantees for excess surrogate ambiguity risk. The authors also formulate the learning task as a constrained optimization problem, analyze its computational complexity in the convex setting, and empirically evaluate the approach against a competing method on several datasets.

By Mohammadreza M. Kalan, Yuyang Deng, Sanaz Hamidi
Hugging Face Trending Papers
Oct 5

Dynamic Minimax Regret Optimization for Robust LLM Post-Training

The paper introduces DUCB-OGD, an algorithm that couples a Discounted Upper‑Confidence‑Bound sampler with Online Gradient Descent to address dynamic minimax regret in robust large‑language‑model post‑training. It operates under instantaneous mini‑batch‑only bandit feedback, tracking worst‑source performance without re‑evaluating historical data. Experiments on fine‑tuning, preference optimization, and reinforcement learning demonstrate that DUCB‑OGD improves worst‑group robustness with negligible computational overhead.