LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?

Judge Arena: Benchmarking LLMs as Evaluators

Evidence First, Arithmetic Second: A System Report and Failure Analysis for DocSem

arXiv:2609.39013v1 Announce Type: new Abstract: EVICALC, our system for the DocSem shared task, achieved 8.61% joint accuracy on 1,730 tasks in the official final test evaluation. It reads a PDF, sel...

What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework

The paper introduces LAST-CQ, a five-agent, training‑free, execution‑grounded framework for Text‑to‑Cypher that evaluates which components of an agentic pipeline contribute most to performance. Experiments on 2,471 live‑database queries across six backbones show that removing correction reduces execution‑BLEU by 3.1–12.3%, while substituting schema‑grounded feedback with raw error strings has negligible impact. Parallel sampling degrades quality by 10–11%, whereas failure detection and retry routing recover 91.7% of initially failed queries, highlighting that simple failure handling is more effective than sophisticated feedback or increased sampling.

MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc

arXiv:2608. 03397v1 Announce Type: new Abstract: MMLongBench-Doc is a long-document QA benchmark of 1,082 questions over 135 PDFs.

Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses

arXiv:2610.02267v1 Announce Type: new Abstract: Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input...

LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents

arXiv:2606. 29399v1 Announce Type: new Abstract: Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters.