The study investigates how language models (LLMs) handle refund or claim requests under written policies when presented with past cases that include both facts and recorded decisions. By separating these two components, the authors found that changing the recorded decisions of near‑copy past cases can increase approval rates by 30–55 percentage points, while altering the facts to support approval actually decreases approval rates, suggesting that the request appears weaker when compared to such facts. These effects were consistent across multiple models and both study‑specific and publicly paraphrased policies.
arXiv:2610.17643v1 Announce Type: new Abstract: An enterprise agent that answers under a written policy must choose an action before it says anything: answer, ask for a missing fact, ask for confirma...
The paper investigates how short memory representations of large‑language‑model (LLM) group discussions affect final decision quality. It introduces CrossProfile, a benchmark of 480 hidden‑profile tasks that separates supporting, decoy, and irrelevant facts while keeping their wording constant. Experiments with Qwen2.5‑7B, Llama‑3.1‑8B, and DeepSeek models show that heuristics based on rarity or relevance both fail in predictable ways, and that the decision model’s social‑proof bias further degrades accuracy. Removing attribution tags or randomizing them improves performance, yet natural transcripts still lose more evidence than no notes at all.
arXiv:2607. 16019v1 Announce Type: new Abstract: AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations.
ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.
arXiv:2606.07834v2 Announce Type: replace-cross Abstract: Evidence-grounded fact verification increasingly uses LLM judges to turn evidence into terminal verdicts. Many task contracts deliberately in...
arXiv:2608. 07813v1 Announce Type: new Abstract: An LLM judge deployed inside a reasoning pipeline does not merely measure quality, it decides which answer ships.
arXiv:2610.27745v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judgment assistants and as stand-in participants in psychology experiments, yet little is known...
arXiv:2610.08319v1 Announce Type: new Abstract: In proof assistants such as Lean, a generated proof must pass machine compilation checks, so evaluation needs no human scoring. Direct generation fails...
The paper presents a clinician‑led safety guardrail for mental‑health chatbots that replaces typical prompt‑engineering or Constitutional AI methods. Two clinicians design, finalize, and adjudicate a rule base of four safety checks, which is formally verified for internal consistency and applied to LLM outputs. The verification layer’s verdicts are recomputable from a named rule and sixteen facts, and it consistently avoids false crisis‑resource demands that a matched LLM judge frequently raises.
Large language models (LLMs) are increasingly used as judgment assistants and as stand-in participants in psychology experiments, yet little is known about how much their probability estimates move in...
arXiv:2607. 23386v1 Announce Type: new Abstract: We document a failure class in frontier large language models -- exception chain collapse -- observed in eligibility evaluation under nested conditional rules of the form "A is required UNLESS B applies, UNLESS C overrides B".
arXiv:2606. 30128v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditioning on more tokens buys extra computation before the model commits to an answer?