arXiv Computation and Language
Two tests of phase-structure features for transition prediction
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Sep 2
Two locked tests of phase-structure features for transition prediction
The article reports on two pre‑specified empirical tests evaluating whether phase‑derived features from a theoretical account of rotary attention improve prediction of commitment or contradiction outcomes. In Study 1, a comparison of the PC‑2 feature against a baseline on 1,136 cases yielded a negligible AUROC difference (+0.00087) that did not meet the pre‑specified threshold, so the advancement was not passed. Study 2 applied fifteen layer treatments to open blocks b0‑b4 across 1,415 transitions, but none met the locked conjunctive rule, resulting in no advancement; the official selection remains null while the theoretical paper is retained.
arXiv Machine Learning
Sep 17
No Usable Linear "Capitulation Direction" in Two Small LLMs: A Validation Protocol for Activation-Steering Claims, and a Cross-Family Behavioral Study of Sycophancy Under Pushback
The study examines how two small instruction‑tuned language models, Qwen2.5‑1.5B and Llama‑3.2‑1B, respond to user pushback on TriviaQA. When initially correct, the models flip to a wrong answer in about 42–43% of cases, with the effectiveness of different pushback styles varying by model. Attempts to decode capitulation from the pre‑response residual stream fail under a rigorous validation protocol, revealing overfitting and a measurement hazard that underestimates capitulation by 18–24 percentage points.
arXiv AI
Oct 2
From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution
Praxa is an evidence‑bound harness for governed AI agent execution that explicitly represents states such as proposal, authority, dispatch, verified external effect, and promotion through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. The authors report four evidence lanes: a repository‑local audit passing all unit and Workerd tests; a pilot on 12 curated tasks where both baseline and reliability‑layer arms passed 17 of 36 trials; a coordination‑proxy comparison where both baseline and a source‑authored candidate completed all 180 trials with equal accuracy but the candidate used fewer tokens and steps; and deployed source/configuration evidence showing bounded reflection, recall accounting, memory compilation, and tool‑health paths. None of the evidence demonstrates superiority in security, safety, or user benefit. whyItMatters:"Praxa provides a testable architecture that makes authority‑to‑effect transitions explicit, offering a framework for verifying AI agent behavior, though current evidence does not prove improved security or performance."
arXiv AI
Sep 11
Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents
The paper introduces Cros, a risk‑constrained stopping layer for sequential clinical diagnosis agents that determines when to stop testing and make a diagnosis. Cros combines state‑wise error ranking, policy design on disjoint development splits, and exact tests of selective diagnostic error to provide finite‑sample guarantees. On a MIMIC‑derived abdominal‑pain benchmark, Cros achieves higher state‑error AUROC and lower selective error rates compared to baseline stopping methods, though its performance varies across development resplits.
arXiv AI
Aug 20
Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow Repair
The study demonstrates that governance records—structured logs linking task contracts, model attempts, verifier decisions, and outputs—can serve as effective supervision for bounded AI models. Using a verifier-selected self‑training approach, the authors show that a Qwen3‑14B model trained on plans accepted by an independent VAL verifier achieved significant gains in plan acceptance across numerous PlanBench replanning cases, outperforming other selection strategies. The results highlight the feasibility of one‑shot execution and cumulative learning without relying on oracle targets or stronger teachers.
arXiv AI
Oct 21
Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims
arXiv:2610.27610v1 Announce Type: cross Abstract: Recursive self-improvement (RSI) promises AI systems, such as self-evolving LLM agents, that become better at producing further improvements. Evaluat...