arXiv AI August 26, 2026 By Heather Renze

Auditing the Synthetic Memoir: Measuring Scene-Level Confabulation in LLM-Generated Autobiography Against the Documented Record of the Life It Describes

The paper audits a 366‑day autobiographical book generated by a large language model (LLM) against an independent verification corpus. Using a four‑level rubric, 354 of the 366 days (96.7%) failed verification, with only 12 days containing corroborated scenes and 19 days containing actively contradicted claims. Regenerating the same days with current models yielded 100% verification failure, while grounding the generation in the subject’s own corpus improved the rate to 83.3% but still left substantial residual failure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

llms

arXiv Computation and Language

Sep 10

When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination

arXiv:2609.09696v1 Announce Type: new Abstract: Large language models are increasingly proposed as automated auditors of document quality, yet their reliability as detectors of planted errors is poor...

By Karan Parekh, Sanjana Pendyala Ravinder, Sana Mhapsekar, Medina Maloku

llms safety

arXiv AI

Aug 13

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

arXiv:2608. 11415v1 Announce Type: cross Abstract: Large language models are being proposed as agents in scientific workflows, in domains where no downstream verifier exists.

By Valentin Rodionov, Shamil Assylbekov

llms agents benchmarks safety

arXiv Machine Learning

Jun 9

Decoy-Calibrated Failure Audits for Language Models

arXiv:2606. 09046v1 Announce Type: new Abstract: Useful audits reveal not only how often a model fails, but also where its failures concentrate.

By Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh

llms benchmarks

arXiv AI

3d ago

Large Language Model Turnover Undermines Screening for Artificial Intelligence-Assisted Scientific Writing

The study examines how the rapid turnover of large language models (LLMs) undermines the effectiveness of AI‑detectors used to screen scientific manuscripts. By comparing 4,000 PNAS abstracts rewritten by 23 different LLM versions released between June 2023 and August 2026, the authors show that detectors trained on past versions can fail dramatically at generation boundaries, missing up to 96% of rewrites just after a sharp transition. Even detectors updated to newer versions can overlook rewrites of older models, and a commercial detector missed most rewrites of the newest version while flagging few human‑written abstracts.

By Kazuki Nakajima, Takayuki Mizuno

llms benchmarks

arXiv Computation and Language

Sep 22

The Copy Ceiling: An Input-Exposure Control for Ontology-Grounded Generation over Curated Corpora

arXiv:2609.24885v1 Announce Type: new Abstract: When a language model answers from a curated corpus via graph-based retrieval, a large grounding uplift does not establish reasoning over the retrieved...

By John J. O'Hare

llms

Hugging Face Trending Papers

Sep 2

VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement

VeriPhy is an auditable physical‑verification system that evaluates generated video by compiling prompts into typed physical obligations and a statically validated execution plan before any frames are observed. During execution, it gates calls to frozen low‑level experts (e.g., segmentation, tracking, counting, depth, OCR, audio‑event detection) and returns provenance‑carrying evidence records, which are mapped to a three‑valued state (supported, contradicted, unknown) with full traceability. On a 1,500‑clip corpus of human‑annotated flaw records, VeriPhy accounts for 228 failures out of 304, outperforming a published question‑decomposition evaluator that accounts for 164, while also providing auditable evidence for each verdict.

llms agents computer-vision