Rome Wasn't Built in One Turn: Replay-Grounded Reward Signals for Long-Horizon Freeciv Agents

The paper introduces CivHarness, a system that records game states in Freeciv and replays them to generate long‑horizon reward signals, addressing the problem of sparse deterministic feedback in strategy games. Using CivHarness, the authors train CivTelescope, an LLM judge that predicts which of two positions will lead to a better outcome, outperforming standard language models on scoreboard‑trap pairs. They then employ CivTelescope as a dense reward in a Freeciv agent, CivMarshmallow, achieving significantly higher training gains and outperforming other reward schemes across the full game.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents