arXiv AI

Embedding Information in Policy Randomness: Distribution-Preserving Implicit Communication in MDPs

arXiv AI
Sep 12

A Mathematical Theory of Pragmatic Information

The paper introduces a pragmatic information theory that unifies communication, control, and decision-making through the isoteleia mapping, which formalizes equifinality by treating distinct semantic paths that lead to the same optimal action as pragmatically equivalent. It establishes a three-tier hierarchy of syntactic, semantic, and pragmatic information, defines pragmatic entropy, mutual information, channel capacity, and rate-distortion, and proves coding theorems that generalize Shannon’s results. The authors also present pragmatic value and cost of information, a Lagrangian dual framework for cross-layer optimization, and a pragmatic efficiency bound that quantifies the maximum net utility for resource-constrained intelligent systems, extending the theory to continuous messages and dynamic settings.

By Kai Niu, Ping Zhang
arXiv AI
Sep 2

Provably Efficient Federated Reinforcement Learning with Linear Function Approximation and Logarithmic Communication Cost

The paper introduces Fed‑LSVI, a federated online reinforcement learning algorithm that uses linear function approximation in episodic Markov decision processes. It achieves a regret bound of ≥O(√{Md^3H^4T}) while only exchanging compressed sufficient statistics, thereby meeting privacy constraints. The method reduces communication cost to logarithmic in the number of episodes, a marked improvement over previous approaches that required linear communication.

By Zihang Liang, Haochen Zhang, Lingzhou Xue
arXiv AI
Sep 3

Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems

The paper introduces a game-theoretic framework for coordinating multi-agent large language model (LLM) systems, modeling orchestrator–worker interactions as a bilevel coordination game. It analyzes textual reflection as stochastic movement over semantic memory states, providing finite-time bounds, lower bounds, and an information-theoretic impossibility result. Building on these insights, the authors propose Stochastic Reflective Memory Ascent (SRMA), a grounded evaluation mechanism that guarantees convergence under certain conditions and demonstrates improved performance on 500 SWE‑bench instances.

By Yihang Chen, Yuxiang Chen, Yuxuan Huang, Meng Fang, Weilin Luo, Jun Wang
arXiv Machine Learning
Sep 14

Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics

The paper investigates learning Nash equilibria in partially observable Markov games (POMGs) where agents cannot fully observe the state. By focusing on a subclass with independent state transitions and a Markov potential game structure, the authors propose an independent learning algorithm that allows agents to converge to an approximate Nash equilibrium using only their own observations and actions, without communication. Under a filter stability assumption, finite‑history policies are shown to approximate the POMG sufficiently, enabling a surrogate near‑potential Markov game and yielding quasi‑polynomial sample and computational complexity.

By Philip Jordan, Maryam Kamgarpour
arXiv Machine Learning
Jun 16

Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process

arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.

By Jongmin Lee, Ernest K. Ryu, Vaneet Aggarwal