The paper introduces a benchmark for predicting a program’s final output and checkpoint state using only its source code and input, extending CRUXEval-style tasks with both short‑ and long‑trace inputs and checkpoints inside and after loops. It evaluates 400 cases from 371 Python and C++ programs across seven settings from four model families, achieving up to 93.0% accuracy on short‑trace final output and 77.0% on long‑trace final output, with reasoning‑enabled settings outperforming non‑reasoning ones by 33.1 to 55.2 percentage points. The study also shows that longer‑trace inputs can change correctness outcomes and that the benchmark reveals errors that short‑output scores alone would miss.
arXiv:2608. 10145v1 Announce Type: new Abstract: LeWorldModel trains a latent world model with a prediction loss and a single anti-collapse regulariser, and reports approximately 87% of goals reached on TwoRoom, its simplest diagnostic environment.
arXiv:2609.36569v1 Announce Type: cross Abstract: Checkpoint selection is a routine decision in supervised fine-tuning (SFT): training produces multiple checkpoints, but only one is retained. Yet fix...
The paper investigates whether the latent states reconstructed during inference match those stored during training in temporal models, focusing on State Discovery in World Machine. By comparing stored state tables with rollouts from the best‑validation checkpoint and the last‑epoch weights, the authors find that reloading the checkpoint used to write the table eliminates 90–91% of the mean per‑dimension Wasserstein‑1 distance, explaining most of the observed state gap. Causal distillation and frozen SD updates fail to meet their criteria under either checkpoint.
Temporal models that maintain per-sequence latent states during training must reconstruct those states when the stored table is unavailable at inference. This creates a basic consistency question: do...
arXiv:2609.39934v1 Announce Type: cross Abstract: Checkpoint selection in domain generalization often relies on source-validation accuracy, yet the selected checkpoint need not provide reliable proba...
The paper investigates how a probe can decode in‑context bindings on model errors and how probe‑guided steering can repair them. It tracks probe accuracy, model output, and steering response across public pretraining and post‑training checkpoints, noting that probe accuracy improves during Pythia pretraining and that steering benefits grow with model size. The study also shows that decoders trained on final state or candidate logits do not outperform each other on late‑checkpoint errors, and presents an information‑theoretic counterexample explaining why decodability on errors alone cannot prove discarded output information.
arXiv:2610.14146v1 Announce Type: new Abstract: Long-horizon terminal benchmarks measure whether an agent can sustain progress through extended workflows with process rewards, not only whether its fi...
arXiv:2607. 26041v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks.
arXiv:2603.01209v3 Announce Type: replace Abstract: In CodeAct, language-model agents write Python that calls tools and use execution feedback to choose actions. Persistent runtimes preserve Python v...
The paper investigates whether hidden-state probes can identify the type of reasoning failure and the specific step that causes it in large language models. By comparing probes trained on final-answer correctness with those trained using Monte Carlo rollouts, the authors find that standard probes miss a large proportion of recoverable failures and that probe performance depends heavily on training design. Across 4B and 8B models, no probe outperforms a simple rule based on trace length for locating the failing step.
ReplaySSM is a technique that redefines how state-space models (SSMs) handle their recurrent state by caching SSM inputs instead of the state itself. It unrolls the recurrence into a checkpoint plus a buffer of recent inputs, reducing state memory traffic by about half and enabling efficient speculative rollback. The method also allows separate precision levels for checkpoints, improving output accuracy while accelerating decoding in hybrid models such as Mamba-2 and Gated DeltaNet.
TRACE (Temporal Audit and Condition-aware Evaluation) is a new benchmark and evaluation framework for streaming video understanding that explicitly records when evidence becomes valid, how visual history is maintained, and how responses are triggered. It combines temporally audited visual tasks, evidence timing, instruction-dependent trigger annotations, a unified causal Core–Adapter protocol, and multidimensional reporting of answer quality, timeliness, response-selection behavior, workload, completion, and reliability. Using 1,240 records from 517 videos, TRACE evaluated eight publicly available models in eight configurations, revealing that similar QA accuracy can hide significant differences in completion, answer validity, generation workload, and proactive performance metrics such as response delay, false alarms, and missed target windows.