arXiv:2608. 10101v1 Announce Type: cross Abstract: Code review is credited with substantially changing a patch's code between its first submission and the version that eventually lands.
arXiv:2604.18835v2 Announce Type: replace-cross Abstract: We propose a scalable, multifactorial experimental framework that systematically probes LLM sensitivity to subtle semantic changes in pairwis...
arXiv:2608. 10216v1 Announce Type: cross Abstract: Agent frameworks ship quality gates that compare text blocks by embedding-cosine similarity and decide at a fixed cutoff.
arXiv:2607. 13111v1 Announce Type: cross Abstract: Distinguishing semantic-preserving commits from changing ones remains an open challenge in software repository mining.
arXiv:2607. 29516v1 Announce Type: cross Abstract: AI coding agents are generating code at volumes that exceed the capacity of traditional peer review.
arXiv:2608.23001v1 Announce Type: new Abstract: Automated manuscript pipelines often regenerate an entire section to repair a local defect, allowing unrelated metrics and citations to change even whe...
The paper introduces MCR-Bench, a benchmark for realistic multi‑round code review that includes 2,269 real‑world tasks across five programming languages, each annotated with fine‑grained defect information and dynamic state labels. Experiments with mainstream large language models show limited overall performance, especially as interaction rounds increase, and reveal that model accuracy varies by defect type and severity. Error analysis identifies key failure mechanisms such as cross‑round temporal misalignment and insufficient long‑range memory.
The paper introduces EquiReview‑R, an AI‑assisted review system that treats omission and over‑critique as distinct risks and refines a structured concern set using evidence‑linked reasoning. It demonstrates that more criticism does not guarantee a better review, showing that many high‑recall reviews lack definitive evidence for concerns and that revision before further search is essential. On a held‑out set of papers, EquiReview‑R meets non‑inferiority for major omission, cuts major over‑critique from 15.5 % to 8.1 %, and stops on 52.4 % of papers, with gains attributed to revision rather than extra inference.
FLAWS is a new benchmark that tests large language models’ ability to find and pinpoint errors that invalidate key claims in scientific papers. The dataset contains 713 paper‑error pairs created by inserting claim‑invalidating mistakes into peer‑reviewed articles using LLMs, and it includes an automated metric for measuring error detection and localization. Five state‑of‑the‑art LLMs were evaluated, with GPT‑5 achieving the highest accuracy of 39.1% when selecting the top 10 candidate error texts.
arXiv:2607. 16019v1 Announce Type: new Abstract: AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations.
arXiv:2608. 19385v1 Announce Type: new Abstract: Historical Arabic manuscript transcription is not only a recognition problem.
The paper investigates whether knowledge editing truly erases original facts from language models. Using a linear trace probe, the authors find that after editing a fact in GPT‑2‑XL, the original object remains highly decodable from hidden states across three different editing methods, even when the model behaves correctly on edited prompts. This suggests that editing suppresses rather than removes the original association in representational space.
arXiv:2609.14738v1 Announce Type: new Abstract: Automated reviewing systems are increasingly evaluated based on the quality of the reviews they produce. Yet a review is only useful if acting on it le...