TreasuryQA Bench is a new benchmark that tests AI agents on real-world tasks involving searching, reading, and extracting exact figures from scanned U.S. Treasury PDFs stored on a shared drive. The benchmark presents 70 questions over 1,435 scanned documents, requiring agents to locate the relevant file via Dropbox’s Model Context Protocol, OCR the necessary pages, and return a precise numeric answer. Results from ten agents show a wide performance gap, with the best agent solving 64.8% of tasks and the worst only 16.7%, and scan quality is shown to significantly impact success rates.
arXiv:2608. 06305v1 Announce Type: new Abstract: Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query.
KnowForge is a modular pipeline for open‑domain multimodal document question answering that separates retrieval, judging, and reading into distinct agents. It uses a hybrid‑retrieval knowledge‑point index, a Locator agent for searching, a Bridge Judge to enforce a second hop, a Deep Reader that handles text, images, and tables, and a synthesizer with format checking. On the M3DocVQA benchmark it achieves an F1 of 47.5, outperforming prior systems by 11 points, and shows strong gains on MultiHop‑RAG and MMLongBench‑Doc as well.
CompLit is a new literature search benchmark covering all eight arXiv domains, designed to evaluate agents on three diagnostic settings: implicit, cumulative, and unmet needs. The benchmark shows that top agents like Codex and Claude Code perform well on standard queries (86‑87%) but drop to 66‑68% on implicit queries, lose additional accuracy when requirements accumulate, and abstain on up to 53% of unmet queries. The study finds that agents often locate the correct or near‑correct paper but misjudge it, indicating a need for more reliable search capabilities.
arXiv:2609.23953v1 Announce Type: new Abstract: AI agents that carry a multi-step computer task through on their own became ordinary tools in the past year, and the same autonomy is available to anyo...
arXiv:2608. 07527v1 Announce Type: cross Abstract: Long-document understanding requires models to find and combine evidence across many pages, layouts, tables, figures, and charts.
arXiv:2609.14412v1 Announce Type: new Abstract: Deep research agents answer complex questions through iterative loops of searching, reading, and reasoning. Recent work on reasoning-intensive benchmar...
AtomCite is an agentic framework that verifies and corrects page‑level citations in multi‑page documents by parsing answers into claims, checking each claim against the cited page image, and applying a deterministic repair policy. The authors introduce DocCite, the first benchmark for this task, built on MP‑DocVQA and DUDE, containing 928 injected instances and 1,909 verified natural errors. Across Gemini, Claude, and GPT models, AtomCite achieves about 93% verification accuracy and improves citation precision from 34% to 87‑90%, while also enhancing hallucination detection in open‑source models.
arXiv:2605. 17554v2 Announce Type: replace Abstract: Frontier deep research agents (DRAs) plan a research task, synthesize across documents, and return a structured deliverable on demand.
arXiv:2607. 28229v1 Announce Type: cross Abstract: The web is increasingly accessed by AI agents rather than humans.
arXiv:2610.29812v1 Announce Type: cross Abstract: Literature-searching language agents are becoming a standard component of scientific assistants for the life sciences, yet there is little controlled...
StagedWorkspace is a versioned workspace designed for knowledge‑work AI agents, ensuring that every parsed view, native file edit, and review diff is explicitly tied to a specific version of the workspace state. By binding parsed records and review diffs to content hashes of native files, the system improves performance on tasks such as OfficeQA and APEX‑Agents, achieving higher pass rates and rubric scores compared to single‑view approaches. The study demonstrates that providing dual parsed/native access and visible diffs enhances agent performance, highlighting workspace state as a key experimental variable for future benchmarks.
arXiv:2608.03617v2 Announce Type: replace-cross Abstract: The personal archive of Konstantin Tsiolkovsky (1857-1935) is held as fond 555 of the Archive of the Russian Academy of Sciences. The archive...