arXiv Computation and Language

Two tests of phase-structure features for transition prediction

arXiv Computation and Language
Sep 2

Two locked tests of phase-structure features for transition prediction

The article reports on two pre‑specified empirical tests evaluating whether phase‑derived features from a theoretical account of rotary attention improve prediction of commitment or contradiction outcomes. In Study 1, a comparison of the PC‑2 feature against a baseline on 1,136 cases yielded a negligible AUROC difference (+0.00087) that did not meet the pre‑specified threshold, so the advancement was not passed. Study 2 applied fifteen layer treatments to open blocks b0‑b4 across 1,415 transitions, but none met the locked conjunctive rule, resulting in no advancement; the official selection remains null while the theoretical paper is retained.

By Abraham Chachamovits
arXiv Machine Learning
Sep 17

No Usable Linear "Capitulation Direction" in Two Small LLMs: A Validation Protocol for Activation-Steering Claims, and a Cross-Family Behavioral Study of Sycophancy Under Pushback

The study examines how two small instruction‑tuned language models, Qwen2.5‑1.5B and Llama‑3.2‑1B, respond to user pushback on TriviaQA. When initially correct, the models flip to a wrong answer in about 42–43% of cases, with the effectiveness of different pushback styles varying by model. Attempts to decode capitulation from the pre‑response residual stream fail under a rigorous validation protocol, revealing overfitting and a measurement hazard that underestimates capitulation by 18–24 percentage points.

By Saad Aamir, Muhammad Awais Bin Adil
arXiv AI
Oct 2

From Proposal to Verified Effect: Praxa, an Evidence-Bound Harness for Governed AI Agent Execution

Praxa is an evidence‑bound harness for governed AI agent execution that explicitly represents states such as proposal, authority, dispatch, verified external effect, and promotion through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. The authors report four evidence lanes: a repository‑local audit passing all unit and Workerd tests; a pilot on 12 curated tasks where both baseline and reliability‑layer arms passed 17 of 36 trials; a coordination‑proxy comparison where both baseline and a source‑authored candidate completed all 180 trials with equal accuracy but the candidate used fewer tokens and steps; and deployed source/configuration evidence showing bounded reflection, recall accounting, memory compilation, and tool‑health paths. None of the evidence demonstrates superiority in security, safety, or user benefit. whyItMatters:"Praxa provides a testable architecture that makes authority‑to‑effect transitions explicit, offering a framework for verifying AI agent behavior, though current evidence does not prove improved security or performance."

By Stefan G. Creadore
arXiv AI
Sep 11

Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents

The paper introduces Cros, a risk‑constrained stopping layer for sequential clinical diagnosis agents that determines when to stop testing and make a diagnosis. Cros combines state‑wise error ranking, policy design on disjoint development splits, and exact tests of selective diagnostic error to provide finite‑sample guarantees. On a MIMIC‑derived abdominal‑pain benchmark, Cros achieves higher state‑error AUROC and lower selective error rates compared to baseline stopping methods, though its performance varies across development resplits.

By Yuexin Wu, Vasile Rus
arXiv AI
Aug 20

Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow Repair

The study demonstrates that governance records—structured logs linking task contracts, model attempts, verifier decisions, and outputs—can serve as effective supervision for bounded AI models. Using a verifier-selected self‑training approach, the authors show that a Qwen3‑14B model trained on plans accepted by an independent VAL verifier achieved significant gains in plan acceptance across numerous PlanBench replanning cases, outperforming other selection strategies. The results highlight the feasibility of one‑shot execution and cumulative learning without relying on oracle targets or stronger teachers.

By Jesus Salas
arXiv AI
Oct 16

Learning When to Withdraw: Curriculum Coverage and Rule-Specific Failures in Fine-Tuned Language Agents

The paper investigates how fine‑tuned language agents can selectively withdraw actions when evidence changes or authority is revoked. Using a synthetic NAQD‑Env environment, the authors compare an original curriculum with an enriched one across multiple system‑prompt framings and evaluate four LoRA checkpoints of Qwen2.5‑3B‑Instruct against instruction‑tuned baselines. Results show that fine‑tuned checkpoints achieve higher decision agreement and withdrawal recall, reduce incorrect withdrawals, and narrow prohibited combinations, though they also omit valid event reports and exhibit mixed preservation of authorized actions.

By Mohamed Abouzahra