arXiv Machine Learning
Jul 17
Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation
arXiv:2606. 14530v3 Announce Type: replace Abstract: Large language models encode rich information in their hidden states.
Code Correctness Signals in LLM Hidden States: Pre-Generation Probing and Repair Geometry
arXiv:2606. 14530v1 Announce Type: new Abstract: Large language models encode rich information in their hidden states.
Related stories
arXiv Computation and Language
Aug 31
Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
arXiv AI
Aug 3
Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?
arXiv:2607. 28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect.
arXiv AI
Aug 18
Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency
arXiv:2608. 16003v1 Announce Type: new Abstract: Automated checking pipelines increasingly place one language model as the checker and another (or the same one) as the fixer.
arXiv Computation and Language
Sep 22
When Residualization Helps an Audit: Format Effects, Slice Gains, and Their Limits
arXiv:2609.24194v1 Announce Type: new Abstract: Evaluation scores used around LLM systems -- including reward models, rerankers, and LLM judges -- can track surface form instead of the quality they c...
arXiv Machine Learning
Sep 11
Legible Failures: Detecting and Repairing In-Context Binding Errors
The paper investigates in-context binding errors in language models, showing that a linear probe can recover correct entity bindings from frozen hidden states even when the model outputs incorrect bindings. Across 16 checkpoints, the probe’s accuracy on failure cases surpasses a baseline by about 0.196, and a probe‑based score improves failure detection over the model’s confidence by 0.079 AUROC. Steering the residual stream toward the probe‑decoded binding further boosts accuracy by an average of 0.168 across eight models.
arXiv Computation and Language
Sep 25
Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax
The paper introduces a three-level evaluation framework—behavioral deployment, LM-head readout, and probe recoverability—to distinguish whether a language model fails a syntactic test by not encoding structure or by failing to use it. Using a trilingual control-dependency benchmark, the authors find that probe recoverability consistently exceeds LM-head readout, which in turn exceeds behavioral deployment across seven models and three languages, with the largest gap observed in Qwen3-0.6B Instruct. Layer-localized activation patching shows that instruction tuning shifts the decoded layer later, suggesting decoding favors surface shortcuts and that behavioral evaluation understates what models encode while probing alone overstates what they deploy.
arXiv Machine Learning
Jun 30
Internal-State Probes Read the Situation, Not the Action: Three Negative Results for Pre-Action Misalignment Monitoring
arXiv:2606. 30449v1 Announce Type: new Abstract: Probes on model internals could help monitor agentic systems if they identify harmful text or tool actions before those actions are generated.
arXiv Computer Vision
2d ago
Encoded but Disconnected: Decomposing Vision-Language Model Failures under a Patching Null
arXiv:2610.00024v1 Announce Type: new Abstract: Across three vision-language model architectures (LLaVA-1.5-7B, Qwen2.5-VL-7B, InternVL3-8B), we report a universal negative finding for mid-layer inte...
arXiv AI
Jul 15
Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models
arXiv:2607. 12962v1 Announce Type: cross Abstract: Frozen small code LLMs are deployed locally, yet the information guiding a retry after a failed attempt is still measured without placebo controls in the self-repair literature.
arXiv Machine Learning
Jun 9
Decoy-Calibrated Failure Audits for Language Models
arXiv:2606. 09046v1 Announce Type: new Abstract: Useful audits reveal not only how often a model fails, but also where its failures concentrate.
arXiv Computation and Language
Sep 15
Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return
arXiv:2609.14758v1 Announce Type: cross Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply...