arXiv AI By Parsa Mazaheri, Kasra Mazaheri

Prior Audit-Repair Context Shifts LLM Verifier Thresholds Toward Leniency

Read the original on arXiv AI →

arXiv:2608. 16003v1 Announce Type: new Abstract: Automated checking pipelines increasingly place one language model as the checker and another (or the same one) as the fixer.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 17

Beyond Accuracy: How Procedural Traces Shift the Decision Criterion of LLM Overseers

The paper investigates how procedural traces—detailed step-by-step accounts of a language model’s reasoning—affect the decision-making of LLM overseers tasked with auditing another model’s outputs. Using signal detection theory, the authors evaluated five overseers on 19 compliance tasks, finding that while error detection remains high when disconfirming evidence is always visible, more elaborate traces shift the decision criterion toward rejection, leading to increased false alarms. The study also shows that providing option labels reduces the stated inability to link evidence to options, yet some overseers still exhibit residual rejection of correct work that grows with trace detail.

By Zihan Chen, Di Zhu, Lei Zheng, Weiling Li