arXiv AI
Jun 11
Planning under Distribution Shifts with Causal POMDPs
arXiv:2602. 23545v2 Announce Type: replace Abstract: In the real world, planning is often challenged by distribution shifts.
A note on goal-based hierarchical RL
The paper discusses the agent-centric general value function (ACGVF) framework, which allows an agent to decide both which goal to pursue and when to consider a goal finished, beyond merely selecting actions. It notes that ACGVF assumes full observability, while a prior approach used an internal belief state but required externally supplied goals. The note proposes to unify and extend these methods using hierarchical hidden Markov models (HHMMs).
Related stories
arXiv AI
Jun 18
What Must Generalist Agents Remember?
arXiv:2606. 18746v1 Announce Type: new Abstract: This paper develops a formal account of what generalist agents must store in memory in order to act near-optimally across multiple environments and goals.
arXiv AI
Jul 28
Action-Sufficient Goal Representations
arXiv:2601. 22496v2 Announce Type: replace-cross Abstract: In offline goal-conditioned reinforcement learning (GCRL), hierarchical approaches decompose long-horizon tasks into high-level subgoal prediction and low-level action execution.
arXiv AI
Jul 23
Sophisticated Policies from Epistemic Priors
arXiv:2607. 19518v1 Announce Type: new Abstract: Sophisticated Inference is a variant of active inference often associated with recursive belief modeling and tree search.
arXiv AI
3d ago
Belief-Based Maximum Occupancy Principle and Active Inference
arXiv:2609.39342v1 Announce Type: cross Abstract: Intrinsic motivation plays a central role in adaptive and goal-directed behavior by conferring agents reward-independent objectives and biases useful...
arXiv AI
Sep 4
Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
The paper introduces Imagine-then-Plan (ITP), a framework that lets agents learn by interacting with a learned world model to generate multi-step imagined trajectories. ITP features an adaptive lookahead mechanism that balances ultimate goals with task progress, producing richer signals about future outcomes. Experiments on various benchmarks show that ITP outperforms existing baselines, and analyses suggest the adaptive lookahead improves reasoning for complex tasks.
arXiv AI
Sep 11
Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability
The paper introduces the Belief-State Engine (BSE), an inference module that supplies a large language model (LLM) with a Bayesian posterior over hidden states in a partially observable Markov decision process (POMDP). By keeping the raw action‑observation log hidden from the LLM, the BSE ensures the agent behaves as a sound Markov policy on the belief MDP, thereby inheriting classical POMDP optimality guarantees. Experiments on the Tiger POMDP and a red‑team attack‑graph task show that BSE‑augmented agents outperform six baselines in task return, belief calibration, and decision consistency.
arXiv AI
Jun 12
The Theory of Mind Utility: Formal Specification of a Mentalizing Mechanism
arXiv:2606. 12721v1 Announce Type: new Abstract: Inferring others' beliefs requires more than reading surface signals; it requires tracking who told them what, in what order, and how credibly.
Hugging Face Trending Papers
3d ago
Belief-Based Maximum Occupancy Principle and Active Inference
Intrinsic motivation plays a central role in adaptive and goal-directed behavior by conferring agents reward-independent objectives and biases useful to act in noisy and uncertain environments. Active...
arXiv AI
Jul 22
Toward Learning POMDPs Beyond Full-Rank Actions and State Observability
arXiv:2601. 18930v4 Announce Type: replace-cross Abstract: We are interested in enabling autonomous agents to learn and reason about systems with hidden states, such as locking mechanisms.
arXiv AI
2d ago
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
The paper introduces Generalized Implicit Temporal Abstraction (GITA), a method for goal-conditioned reinforcement learning that conditions a single value function on multiple temporal abstraction levels (k). By aggregating advantage-weighted supervision across various k values, GITA preserves both long-range signal and local resolution without committing to a single k. Experiments on OGBench show that GITA outperforms existing offline GCRL baselines, improving average success rates by 25 percentage points over HIQL and 7 percentage points over OTA.
arXiv AI
Jul 15
Calculating Mutual Information between a Reward Maximizer and its Environment
arXiv:2602. 12963v2 Announce Type: replace Abstract: An important question in the field of AI is the extent to which successful behaviour requires an internal representation of the world.