← Back to all news
arXiv Computation and Language By Junwei Yao, Zike Yang, Yongxi Zhou, Yanxian Chen, Jianan Liu, Jiaxi Wen, Zhijun Wang

Precedent Labels and Facts Shift LLM Approval Decisions in Opposite Directions

Read the original on arXiv Computation and Language →

The study investigates how language models (LLMs) handle refund or claim requests under written policies when presented with past cases that include both facts and recorded decisions. By separating these two components, the authors found that changing the recorded decisions of near‑copy past cases can increase approval rates by 30–55 percentage points, while altering the facts to support approval actually decreases approval rates, suggesting that the request appears weaker when compared to such facts. These effects were consistent across multiple models and both study‑specific and publicly paraphrased policies.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Oct 28

Rare, Relevant, and Still Wrong: How Memory Fails LLM Group Decisions

The paper investigates how short memory representations of large‑language‑model (LLM) group discussions affect final decision quality. It introduces CrossProfile, a benchmark of 480 hidden‑profile tasks that separates supporting, decoy, and irrelevant facts while keeping their wording constant. Experiments with Qwen2.5‑7B, Llama‑3.1‑8B, and DeepSeek models show that heuristics based on rarity or relevance both fail in predictable ways, and that the decision model’s social‑proof bias further degrades accuracy. Removing attribution tags or randomizing them improves performance, yet natural transcripts still lose more evidence than no notes at all.

By Anouar Imel, Imad Charradi, Abderrahman Sghir, Khalil El Khammar
llmsagentsbenchmarks
More like this →
arXiv AI
Oct 16

ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing

ClaimScope is a 72‑task benchmark that audits mathematical claims by requiring a verdict, scope, counterexample, and repair, covering areas such as optimizer‑state transport and exact $2 imes2$ linear algebra. The study evaluates Qwen3‑8B using a witness checker that only examines the model’s submitted state, showing that verifier feedback improves counterexample acceptance and safe completions compared to generic feedback. The results demonstrate that separating claim classification from executable evidence can yield gains in witness‑structure correction without necessarily improving broader reasoning.

By Mushir Akhtar
agentsbenchmarks
More like this →