The paper compares a non‑generative verifier (Jev) with a frontier LLM (Astra) on 495 human‑labelled claims from LLM‑AggreFact. Both models achieve statistically indistinguishable accuracy, yet Jev is 187‑fold cheaper and admits more unsupported claims. A confidence‑gated cascade that escalates uncertain claims improves cost‑efficiency, though the benefit comes mainly from the escalation share rather than the probability estimates.
By Alden Do Rosario, Kirollos Atef
arXiv:2607. 23976v1 Announce Type: cross Abstract: Appending a two-word confirmation tag to a decision question -- "Is X the better choice?
By Tapan Parikh
arXiv:2610.32955v1 Announce Type: new
Abstract: Personal email agents decide, for every message, what it is, whether it is urgent, whether to draft a reply, create a task, or ignore it. Their error r...
By Federico Lucca
arXiv:2606. 01435v1 Announce Type: new Abstract: LLM-based memory systems increasingly maintain facts that evolve over time, where a recurring failure is conflict resolution: when a fact has multiple contradictory values, which should the agent return?
By Vikas Reddy, Sumanth Challaram
arXiv:2610.02267v1 Announce Type: new
Abstract: Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input...
By Jiawei Li
The paper evaluates System One decision models—typed models that output probabilities for branching decisions—against supervised classifiers and generative language models on automated decision gate tasks. Eight checkpoints from six families, including the hosted model Jev, were benchmarked on workflow, intent, and social‑science items, showing that small trained classifiers match or slightly outperform decision models on intent and workflow when labels are available, while decision models outperform zero‑shot classifiers when labels are absent. The study also explores calibration, risk thresholds, and cost‑efficiency trade‑offs, providing condition‑dependent design guidelines for automated decision gates.
By Amir Rafe, Subasish Das
Appending a two-word confirmation tag to a decision question -- "Is X the better choice? " versus "X is the better choice, right?
arXiv:2610. 29052v1 Announce Type: cross Abstract: A typed decision maps evidence, a question and a fixed set of options to a probability for every option, computed in one forward pass, by reading a language model's logits or a small head over its hidden states, instead of generating and parsing text.
By Yuchuan Weng, Chi Liu, Xin Chen, Srinivasan Manoharan
The paper investigates how machine‑learning models can determine whether a claim (a test assertion) remains valid after a code change. It compares two questioning strategies: asking whether a diff preserves behavior versus asking whether a specific claim still holds. The authors find that the latter approach yields far higher precision (up to 0.974) across models of varying cost, while the former performs poorly (precision 0.291–0.329). They also benchmark against a regression‑test selector, showing that even near‑complete knowledge of a change’s reach does not reliably identify falsified claims. The study is grounded in 10,369 mined claims with 184 execution‑verified flips from 23 Python libraries.
By Atul Anand
The paper investigates how short memory representations of large‑language‑model (LLM) group discussions affect final decision quality. It introduces CrossProfile, a benchmark of 480 hidden‑profile tasks that separates supporting, decoy, and irrelevant facts while keeping their wording constant. Experiments with Qwen2.5‑7B, Llama‑3.1‑8B, and DeepSeek models show that heuristics based on rarity or relevance both fail in predictable ways, and that the decision model’s social‑proof bias further degrades accuracy. Removing attribution tags or randomizing them improves performance, yet natural transcripts still lose more evidence than no notes at all.
By Anouar Imel, Imad Charradi, Abderrahman Sghir, Khalil El Khammar
arXiv:2607. 10202v1 Announce Type: new Abstract: Cross-model comparisons read divergence in value dispositions as evidence that language models hold individuated values.
By Hong-In Won, Jinseok Jang, Hyoseop Kim
Augur is a synthetic decision laboratory that simulates how users will react to product and policy changes before they are released. It constructs a typed knowledge graph from change documents, populates a persona market, runs simulations, and produces an auditable decision memo recommending one of five actions. Using a dataset of 50 real episodes (Gold‑50), the authors evaluate the system’s five‑way release verdicts and find that evaluation design, rather than model capability, largely drives performance differences among frontier and open‑weight models.
By Rahul Khedar, Mayank Malhotra, Avinash Karn