arXiv AI
Jul 13
Minimal Decision Dynamics and Contextual Probability: A Quantum Tug-of-War Model
arXiv:2601. 10034v2 Announce Type: cross Abstract: Decision making often exhibits context dependence that challenges classical probability theory.
Contextual Probability and Operational Minimality in Tug-of-War Decision Making
The paper investigates how minimal decision-making relates to contextual probability using the Tug-of-War (TOW) principle for multi‑armed bandit problems. By fixing reward‑learning and information requirements, the authors show that minimizing relaxation of common event semantics can produce nonclassical probability, deriving the exact minimum semantic repair and conditions for selecting contextual statistics. They further develop a quantum Tug‑of‑War (QTOW) instrument that improves stationary decision performance, surpasses the KCBS noncontextuality bound, and provides a diagnostic for a corrected contextuality witness.
Related stories
arXiv AI
Aug 18
Semantic Bandits: In-Context Exploration-Exploitation is Biased by Semantic Priors
arXiv:2608. 16707v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in settings that require sophisticated environmental exploration.
arXiv Machine Learning
Jun 2
A Direct Approach for Handling Contextual Bandits with Latent State Dynamics
arXiv:2604. 08149v2 Announce Type: replace Abstract: We consider a linear contextual bandit model where contexts and rewards are governed by a finite hidden Markov chain.
arXiv Machine Learning
Aug 13
When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits
arXiv:2608. 11560v1 Announce Type: new Abstract: Personalizing marketing messages with contextual multi-armed bandits (CMABs) drives real business value, yet the objective that ultimately matters - a downstream conversion - is observed only weeks later, too late to drive online learning.
arXiv Machine Learning
Aug 18
Sequential Batch Learning in Finite-Action Linear Contextual Bandits
arXiv:2004. 06321v2 Announce Type: replace Abstract: We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.
Hugging Face Trending Papers
Aug 12
When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits
Personalizing marketing messages with contextual multi-armed bandits (CMABs) drives real business value, yet the objective that ultimately matters - a downstream conversion - is observed only weeks later, too late to drive online learning. Teams therefore train the bandit on a fast proxy reward, and separately must judge whether a contextual bandit is worth its complexity over sending one best message.
arXiv Machine Learning
Sep 10
Mixing Makes Markovian Contexts Cheap for Linear Bandits
The paper extends the idea that contexts are cheap for linear bandits from i.i.d. settings to Markovian context processes. By assuming uniform geometric ergodicity, the authors construct a stationary surrogate action set and use a delayed‑update scheme to mitigate bias from nonstationary conditional context distributions. They provide a phased algorithm for unknown stationary distributions and achieve high‑probability regret bounds comparable to standard linear bandit oracles in fast‑mixing regimes, with empirical validation showing gains over LinUCB.
arXiv Machine Learning
Sep 10
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
The paper introduces state abstractions that preserve the difference of Q‑functions for offline reinforcement learning, aiming to exclude irrelevant dynamics from rich state data. It proposes a dynamic generalization of the R‑learner that uses orthogonal estimation and sparse learning to estimate the Q‑function contrast, achieving faster convergence and consistency under a margin condition. Experiments on simulated and simulator‑augmented real data show variance reductions and demonstrate that the necessary information for sequential decision‑making can be smaller than that required for full state prediction.
arXiv AI
Jul 13
Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
arXiv:2607. 09015v1 Announce Type: cross Abstract: We study contextual bandit problems with correlated arms and access to surrogate reward signals produced by a machine learning model, motivated by applications such as large language model (LLM) routing.
arXiv AI
Jun 19
Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models
arXiv:2606. 19750v1 Announce Type: cross Abstract: Reinforcement learning (RL) is a central approach for improving reasoning capabilities in large language models (LLMs), where training efficiency depends critically on how problems are sampled during optimization.
Hugging Face Trending Papers
Jul 15
Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation
We study the contextual combinatorial semi-bandit (CCSB) problem with general reward function approximation. At each round, the learner observes a context, selects a combinatorial action consisting of a subset of basic arms, and receives the reward of each selected arm; the goal is to maximize the cumulative reward over time.
arXiv Machine Learning
Jun 2
Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanisms
arXiv:2508. 02812v3 Announce Type: replace Abstract: Causal graphical models can encode large amounts structural knowledge, both from the background knowledge of domain experts and the structural knowledge discovered from randomized experiments or observational data.