arXiv Machine Learning
23h ago
Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements
arXiv:2608. 17310v1 Announce Type: new Abstract: Reinforcement Learning (RL) has been promising in single-turn LLM fine-tuning.
BioBlue: Systematic runaway-optimiser-like LLM failure modes on biologically and economically aligned AI safety benchmarks for LLMs with simplified observation format
arXiv:2509. 02655v3 Announce Type: replace-cross Abstract: Many AI alignment discussions of "runaway optimisation" focus on RL agents: unbounded utility maximisers that over-optimise a proxy objective (e.
Related stories
arXiv AI
Jun 18
STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability
arXiv:2606. 19236v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards algorithms like GRPO have emerged as the dominant post-training paradigm for complex reasoning in LLMs, yet commonly suffer from policy entropy collapse during training.
arXiv AI
Aug 3
AIvilization v0: Toward Large-Scale Artificial Social Simulation with a Unified Agent Architecture and Adaptive Agent Profiles
arXiv:2602. 10429v2 Announce Type: replace-cross Abstract: AIvilization v0 is a publicly deployed large-scale artificial society that couples a resource-constrained sandbox with a unified LLM-agent architecture, aiming to sustain long-horizon autonomy while remaining executable under a rapidly changing environment.
arXiv AI
Aug 11
Emergence Invariance: From Symbolized Thought to Structural Control
arXiv:2608. 01548v2 Announce Type: replace Abstract: Language-first intelligence is constrained by which distinctions enter its symbolic record, which mappings its language--interpreter--environment complex can execute, and which possibilities can be realized with finite resources.
arXiv AI
Jul 13
Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization
arXiv:2602. 07764v2 Announce Type: replace-cross Abstract: Multi-objective reinforcement learning (MORL) seeks to train agents capable of balancing conflicting objectives.
arXiv AI
Jun 29
When are LLMs Sufficient Policy Optimizers for Sequential RL Tasks?
arXiv:2605. 30719v2 Announce Type: replace-cross Abstract: We study when large language models (LLMs) can serve as effective black-box policy optimizers for reinforcement learning (RL) tasks, i.
arXiv Machine Learning
Jun 30
Predictive Objectives Discard Exogenous Control-Relevant Features: A Controlled Mechanistic Study
arXiv:2606. 30068v1 Announce Type: new Abstract: Joint-embedding predictive (JEPA-style) objectives learn representations by predicting future latents.
arXiv AI
Jul 7
Regime-Conditional Stabilisation of LLM-Augmented Cooperative Multi-Agent Reinforcement Learning
arXiv:2607. 04470v1 Announce Type: cross Abstract: Large Language Models (LLMs) offer a natural interface for translating human objectives into reward signals for cooperative multi-agent reinforcement learning (MARL), yet the training-time dynamics of this integration remain poorly understood.
arXiv AI
Aug 11
Rethinking Self-Evolving Agents: Do We Still Need Prescribed Optimization Pipelines?
arXiv:2608. 09629v1 Announce Type: new Abstract: Self-evolving agents are usually built around prescribed optimization pipelines: the framework decides how to gather evidence, revise a persistent artifact, select candidates, and stop.
arXiv AI
Jul 16
Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0
arXiv:2607. 14004v1 Announce Type: new Abstract: Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method.
arXiv AI
Jun 24
Beyond Trajectory Imitation: Strategy-Guided Policy Optimization for LLM Reasoning
arXiv:2606. 24064v1 Announce Type: new Abstract: Distilling reasoning capabilities from strong to weak language models typically involves imitating specific solution trajectories, effectively transferring what to answer rather than how to reason.
arXiv Machine Learning
Jul 13
SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions
arXiv:2607. 08925v1 Announce Type: new Abstract: Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do.