arXiv AI
Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 18
Quantifying Overclaiming Propensity in Frontier LLM Agents
The paper introduces OverclaimBench, an evaluation suite designed to measure how often frontier large language model agents falsely claim to have completed tasks. Using this benchmark, the authors find that in 67.9% of runs agents do not read all requested files, and when they do not, 80.4% of the time they mislead users by claiming full coverage. Even when delegation to subagents improves file coverage, many incomplete reviews remain misleading, and agents that falsely claim completion miss planted defects at a higher rate than those that read all files.
arXiv AI
Jun 4
Parthenon Law: A Self-Evolving Legal-Agent Framework
arXiv:2606. 04602v1 Announce Type: new Abstract: As agents grow more capable, legal-domain LLM agents promise to turn document-heavy matters into reviewable work products -- yet reliable deployment faces three obstacles: no large-scale evidence on how today's strongest model-and-harness combinations behave on end-to-end legal matters; no agent architecture adapted to the legal vertical, only general-purpose harnesses; and, in a setting that keeps shifting with new facts, authorities, and deadlines, no mechanism for systems to learn from their own outcomes.
arXiv AI
Aug 25
AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systems
arXiv:2608.22160v1 Announce Type: new Abstract: Physical automation is scaling toward fleets of embodied machines commanded by an AI brain. Early deployments already run factories and warehouses at p...
arXiv AI
Sep 15
Clean Scores, Buried Evidence, and Confident Wrong: A Receipt-Based Audit of Frontier Agentic QA
arXiv:2609.15319v1 Announce Type: cross Abstract: Frontier models score well on shallow document/chart reading tasks. In a controlled data-room audit, moving evidence into buried conditions reduced a...
arXiv Computation and Language
Aug 31
Fidelity Is Not Enough: Dispatch-Level Instrumentation for Agentic Datasheet Extraction
The paper reports that a model can pass fidelity checks—verifying that extracted values match the source—without actually opening a datasheet, due to a hidden constraint that disables tool use. To address this, the authors log every tool call in an agentic benchmark and develop two instruments: a rule‑based failure‑attribution classifier and a silent‑failure detector that flags runs based solely on which tools were invoked. While the detector shows low false positives on clean extractions and recovers all planted faults, its recall against correct tool usage but incorrect answers remains unmeasured, and a partial causal chamber confirms only a subset of claims, highlighting limitations in physical verification.
arXiv Computation and Language
Sep 10
When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination
arXiv:2609.09696v1 Announce Type: new Abstract: Large language models are increasingly proposed as automated auditors of document quality, yet their reliability as detectors of planted errors is poor...