arXiv:2608. 08512v1 Announce Type: new Abstract: Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates.
The paper introduces THPT‑Ladder, a benchmark based on Vietnam’s 2025 National High School Graduation Examination’s convex grading scheme, which rewards partial credit non‑additively. It shows that standard accuracy metrics inflate model scores because they treat partial knowledge proportionally, whereas the official rubric penalizes incomplete correct sets. Using the benchmark, the authors demonstrate that this discrepancy can shift a model’s percentile ranking by up to 13 points among over 480,000 candidates.
arXiv:2607. 16019v1 Announce Type: new Abstract: AI systems increasingly retrieve from records that revise themselves: issue threads, encyclopedic histories, policy logs, and long conversations.
NitiBench+ is a Thai legal question‑answering benchmark that spans 17 domains and contains 1,670 items derived from real court decisions. Each item includes both the law text that was in force at the decision date and the current text, along with a retrieval corpus of every dated version of every Thai statute section. The benchmark evaluates models using the Thai Bar Council rubric, Bar‑standard score, and citation overlap, and it reveals that reading outdated or missing law significantly degrades performance.
arXiv:2610.32191v1 Announce Type: new Abstract: A collection date records when a source revision was acquired, not when its contents changed. Can assigning this metadata redirect temporal question an...
arXiv:2610.19848v1 Announce Type: new Abstract: We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench conta...
The study examines how the rapid turnover of large language models (LLMs) undermines the effectiveness of AI‑detectors used to screen scientific manuscripts. By comparing 4,000 PNAS abstracts rewritten by 23 different LLM versions released between June 2023 and August 2026, the authors show that detectors trained on past versions can fail dramatically at generation boundaries, missing up to 96% of rewrites just after a sharp transition. Even detectors updated to newer versions can overlook rewrites of older models, and a commercial detector missed most rewrites of the newest version while flagging few human‑written abstracts.
The paper investigates how memory systems can answer a current query correctly yet fail to retain distinctions needed for later updates. Using a paired‑history audit, the authors evaluate 24 history pairs across six synthetic mechanisms and two model backends, achieving perfect reveal accuracy on DeepSeek and high accuracy on GLM. Record‑level audits reveal specific failures in structured reveal memories and frontier late‑reference adequacy, and the authors test a label‑equivariant repair that only partially restores correctness.
arXiv:2608. 04160v1 Announce Type: cross Abstract: Multilingual evaluations report accuracy at a single output-token cap, but languages need different numbers of tokens to express the same content, so the cap is a hidden experimental variable.
The paper introduces RoPA Manager, a system that automates the extraction of Records of Processing Activities (RoPA) required by Vietnam’s new Personal Data Protection Law. It combines hybrid retrieval techniques—lexical ranking, dense‑vector search, and Reciprocal Rank Fusion—with locally deployed large language models to avoid data‑sovereignty issues. A Vietnamese RoPA benchmark of 32 organizations and 77 processing activities was created, and the system achieved robust scorer performance (F1 ≈ 0.95) and moderate end‑to‑end token coverage (≈ 50‑55%).
The paper introduces *norregel*, a preregistered benchmark that tests language models on numeric values in Norwegian legal texts, including amounts, percentages, and age limits, each tied to a specific date. The benchmark contains 916 items extracted from Lovdata, allowing answers to be classified as correct, outdated, wrong, or uninterpretable. Two preregistered runs were conducted: the first showed mixed results across five models, while the second, involving eight models, supported the hypothesis that outdated answers are more common for values introduced after a model’s cutoff, though the robustness of this finding varied across sensitivity analyses.
The paper reports that in English all‑words word sense disambiguation (WSD), the scarcity of high‑quality labels—not the models—has become the limiting factor. The authors introduce lexEN, a human‑adjudicated correction layer over the Maru2022 ALL_NEW benchmark, and SenseBench, a living leaderboard for LLM WSD evaluation. They show that frontier large language models reach about 95 % accuracy on lexEN‑v1, that relabeling corpora with these models improves downstream systems, and that fine‑grained WordNet senses are often ill‑posed, with coarsening improving both annotator agreement and model performance. "whyItMatters":"The study highlights that improving label quality and managing annotation costs are now the critical challenges for advancing WSD performance, as model accuracy is already near its theoretical ceiling."