The paper introduces an acceptance protocol for sensor-coordinate and polarity binding in mechatronic commissioning, separating candidate generation from release authority. It routes unsupported requirements to a frozen four‑billion‑parameter local language model and only releases plans when both facts can be derived by an external gate under a sealed grammar, with a gold‑standard user providing a canonical answer when eligible. Evaluation on 144 tasks showed that fabricated plans were correctly rejected, no false releases occurred among 83 releases, and protection against incorrect user answers was characterized across 431 pairings.
The paper argues that large language model agents should not be the sole authority on whether they have satisfied a task. It identifies two gaps—understanding–execution and state–authority—where agents may claim completion without actually meeting the specification. The authors propose SpecHarness, a framework that separates agent proposals from authoritative state by requiring evidence from qualified providers to confirm compliance, and demonstrate its effectiveness on guideline‑following and artifact‑generation tasks.
By Haiqing Li, Xin Ma, Yinhao Wu, Wenliang Zhong, Feng Jiang, Thao M. Dang, Xiao Hu, Hehuan Ma, Yuzhi Guo, Junzhou Huang
arXiv:2609.26048v1 Announce Type: cross
Abstract: Language-model agents often reach a working solution and then fail to consistently deliver it. We study runtime policies: targeted natural-language i...
By Nikita Agarwal, Nivedit Jain
The paper introduces NAQD‑Env, a synthetic benchmark designed to test language agents’ ability to selectively withdraw and resume actions when new evidence, permissions, or stop instructions arise. It evaluates models against a deterministic reference policy across eleven dependency families, measuring policy agreement, task value, withdrawal, resumption, and event reporting. Experiments on 350 scenarios show low withdrawal recall, no valid resumption, and limited policy alignment, highlighting the need to treat selective withdrawal as a distinct reliability component.
By Mohamed Abouzahra
Praxa is an evidence‑bound harness for governed AI agent execution that explicitly represents states such as proposal, authority, dispatch, verified external effect, and promotion through deterministic admission, brokered execution, external read‑back, reconciliation, and reviewed promotion. The authors report four evidence lanes: a repository‑local audit passing all unit and Workerd tests; a pilot on 12 curated tasks where both baseline and reliability‑layer arms passed 17 of 36 trials; a coordination‑proxy comparison where both baseline and a source‑authored candidate completed all 180 trials with equal accuracy but the candidate used fewer tokens and steps; and deployed source/configuration evidence showing bounded reflection, recall accounting, memory compilation, and tool‑health paths. None of the evidence demonstrates superiority in security, safety, or user benefit.
whyItMatters:"Praxa provides a testable architecture that makes authority‑to‑effect transitions explicit, offering a framework for verifying AI agent behavior, though current evidence does not prove improved security or performance."
By Stefan G. Creadore
arXiv:2607. 19449v1 Announce Type: cross Abstract: Evaluation frameworks for tool-augmented LLM agents focus overwhelmingly on capability metrics or explicit tool crashes, leaving silent infrastructure failures and HTTP 200 responses with empty, null, or malformed payloads largely unaudited.
By Aarushi Singh