The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper compares a non‑generative verifier (Jev) with a frontier LLM (Astra) on 495 human‑labelled claims from LLM‑AggreFact. Both models achieve statistically indistinguishable accuracy, yet Jev is 187‑fold cheaper and admits more unsupported claims. A confidence‑gated cascade that escalates uncertain claims improves cost‑efficiency, though the benefit comes mainly from the escalation share rather than the probability estimates.
arXiv:2607. 23976v1 Announce Type: cross Abstract: Appending a two-word confirmation tag to a decision question -- "Is X the better choice?
arXiv:2610.32955v1 Announce Type: new Abstract: Personal email agents decide, for every message, what it is, whether it is urgent, whether to draft a reply, create a task, or ignore it. Their error r...
arXiv:2606. 01435v1 Announce Type: new Abstract: LLM-based memory systems increasingly maintain facts that evolve over time, where a recurring failure is conflict resolution: when a fact has multiple contradictory values, which should the agent return?
arXiv:2610.02267v1 Announce Type: new Abstract: Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input...
The paper evaluates System One decision models—typed models that output probabilities for branching decisions—against supervised classifiers and generative language models on automated decision gate tasks. Eight checkpoints from six families, including the hosted model Jev, were benchmarked on workflow, intent, and social‑science items, showing that small trained classifiers match or slightly outperform decision models on intent and workflow when labels are available, while decision models outperform zero‑shot classifiers when labels are absent. The study also explores calibration, risk thresholds, and cost‑efficiency trade‑offs, providing condition‑dependent design guidelines for automated decision gates.