← Back to all news
arXiv Machine Learning

Decodable In-Context State and Model Output Across Training

The paper investigates how a probe can decode in‑context bindings on model errors and how probe‑guided steering can repair them. It tracks probe accuracy, model output, and steering response across public pretraining and post‑training checkpoints, noting that probe accuracy improves during Pythia pretraining and that steering benefits grow with model size. The study also shows that decoders trained on final state or candidate logits do not outperform each other on late‑checkpoint errors, and presents an information‑theoretic counterexample explaining why decodability on errors alone cannot prove discarded output information.

Related stories

arXiv Machine Learning
Sep 11

Legible Failures: Detecting and Repairing In-Context Binding Errors

The paper investigates in-context binding errors in language models, showing that a linear probe can recover correct entity bindings from frozen hidden states even when the model outputs incorrect bindings. Across 16 checkpoints, the probe’s accuracy on failure cases surpasses a baseline by about 0.196, and a probe‑based score improves failure detection over the model’s confidence by 0.079 AUROC. Steering the residual stream toward the probe‑decoded binding further boosts accuracy by an average of 0.168 across eight models.

By Manas Venkata Sai Ravulapalli, Samrath Singh Chadha, Abhinav M. Hari
llms
More like this →
arXiv Machine Learning
Sep 1

The Intervention Gap in Latent World Models

The paper introduces the concept of intervention fidelity in latent world models, measuring whether a model’s open‑loop transitions align with actual environment interventions. Experiments on TD‑MPC2, Cheetah, and DreamerV3 show that high reward fit does not guarantee fidelity, and that self‑supervised models can outperform task‑anchored ones in preserving intervention effects. The authors propose a capture‑gated audit to localize failures and argue that fidelity must be directly audited on the model’s native interface.

By Donna Vakalis
robotics
More like this →
arXiv AI
Aug 24

Beyond End-to-End Success: Diagnosing Failures in Long-Horizon Security LLM Agents

The paper introduces a diagnostic framework for long‑horizon security LLM agents that uses checkpoints to distinguish failures occurring before and after a model’s capability is exposed, and applies controlled interventions to pinpoint upstream bottlenecks. The methodology is tested on four task families—delayed reuse of discovered information, reuse of observed state, recovery from failed strategies, and decision making after uncertain outcomes—revealing that many failures happen before the agent observes the state it later needs to reuse. Experiments with Gemini 2.5 Flash and Gemini 3.7 Flash show that targeted protocol‑disambiguation guidance can significantly alter state observation rates and that the primary source of failure can shift across model generations, underscoring the need for fine‑grained failure diagnostics rather than relying solely on overall task success.

By Wei Shao, Chongzhou Fang, Zuxiong Tan, Zequan Liang, Setareh Rafatirad, Avesta Sasan, Houman Homayoun
llmsagents
More like this →