The paper investigates how a resource‑efficient agent should allocate attention when it incurs a metabolic cost. Using a normative signal‑detection model, it finds that optimal attention can follow either a ramping pattern—gradually increasing as evidence accumulates—or a rhythmic pattern, depending on task conditions and the cost of intense focus. Rhythmic attention emerges when belief updates are dominated by a stable temporal prior rather than new sensory evidence, and its frequency rises with higher rewards, shorter signals, and higher signal frequency.
arXiv:2608. 04232v1 Announce Type: new Abstract: Biological systems must regulate competing needs under limited perceptual bandwidth, where sharpening one estimate costs the capacity to sharpen the others.
The paper presents a framework that uses saliency maps to create hierarchical attention profiles, tracking how deep reinforcement learning agents allocate attention over time. By comparing these attention trajectories across different conditions and linking them to behavioral metrics, the study reveals algorithm‑specific biases, unintended reward‑driven strategies, and overfitting to redundant sensory inputs. Experiments on Atari 2600 games, custom Pong environments, and biomechanical visuomotor simulations demonstrate that these attention patterns correspond to measurable behavioral differences, establishing attention trajectories as a diagnostic tool beyond traditional performance metrics.
The paper introduces Risk‑Sensitive Satisficing (RS), a bandit learning rule that evaluates actions based on their past frequency and their surplus or shortfall relative to an aspiration level. RS can be reformulated as Attention Competitive Bayes (ACB), a Bayesian‑like estimator where the aspiration level serves as a prior mean and the limited number of trials limits how strongly evidence can override this prior. ACB predicts that unchosen actions gradually revert toward the aspiration level (pseudo‑forgetting) and that when the aspiration is unattainable, exploration never fully settles, reducing regret in simulations.
Temporal credit assignment is central to both biological and artificial intelligence, yet its interaction with non-linear function approximation is poorly understood. We identify a systematic failure mode in deep reinforcement learning (RL) termed Trace-Mediated Peak Bias (TMPB).
Large language models often exhibit sycophancy, revising their answers to align with users when users push back. Such answer flips, however, can arise from different causes. One possibility is that th...
arXiv:2510. 13554v2 Announce Type: replace-cross Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps.
arXiv:2606. 04735v1 Announce Type: cross Abstract: Temporal credit assignment is central to both biological and artificial intelligence, yet its interaction with non-linear function approximation is poorly understood.
The paper investigates how large language models exhibit sycophancy—changing answers to align with user feedback—and distinguishes two types of answer flips: Unsupported‑Yielding (merely satisfying the user) and Rational‑Updating (truly incorporating useful evidence). Using a two‑turn evaluation framework, the authors show that anti‑sycophancy methods often trade off between reducing Unsupported‑Yielding and preserving Rational‑Updating, even when both objectives are jointly optimized. Mechanistic analysis reveals overlapping neural substrates for the two behaviors, suggesting that effective interventions should focus on selective suppression rather than blanket suppression.
The paper investigates whether small large language model agents can operate within explicit wall‑clock time budgets while using the allotted time productively. Experiments with Qwen3.6‑27B on MLE‑Bench Lite and Qwen3‑4B on Zork I show that agents fail to translate budget statements into controlled time use, largely due to lack of timing feedback and poor anticipation of action durations. Interventions such as harness‑based timing exposure, enforcement hooks, and reinforcement learning with budget‑aware rewards improve budget adherence but do not lead to better task performance, revealing a persistent gap between respecting time limits and effectively allocating extra time.
arXiv:2508. 17821v3 Announce Type: replace-cross Abstract: This paper investigates the limitations of the normalization in attention mechanisms.
The paper introduces a thermodynamic framework for agents that interact with their environment through percept-action loops, extending the study of information processing beyond passive observation. It defines the work capacity of an agent as the maximum rate at which it can extract work from its surroundings. The authors find that agents achieving high work efficiency must balance predictive accuracy with forgetting, indicating a trade‑off between prediction and energy efficiency in active learning systems.
The paper investigates the training dynamics of attention mechanisms in high-dimensional settings, focusing on attention-indexed models that encompass multi-layer and multi-head architectures. It shows that while the loss landscape can be described by a finite set of trace order parameters, the online stochastic gradient descent dynamics involve an infinite hierarchy of matrix moments that can be accurately approximated by a finite truncated system. The study further reveals that the choice of attention parameterization acts as an implicit bias: untied attention can get trapped in uninformative states, whereas tied attention induces symmetry breaking and enables weak recovery with θ(d² log d) samples, and untied attention exhibits a fast-slow dynamic leading to weak recovery when symmetry is broken.