Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers

The paper introduces "Trace as State," a method that places collected reasoning traces before a long-context block in a transformer, allowing earlier derived information to guide rereading. Experiments across three models and datasets show that this approach consistently outperforms a control where traces are appended after the context, achieving significant accuracy gains on tasks such as GraphWalks Parents and GLM-5.2. The results demonstrate that positioning traces before the context can enhance long-context reasoning while preserving the causal transformer architecture.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms