arXiv AI
Jul 31
APEX-Accounting
arXiv:2607. 27189v2 Announce Type: cross Abstract: We introduce APEX-Accounting, a benchmark built by Mercor in partnership with Ramp, to assess whether frontier models can do the real work of accountants.
Crypto Accounting Bench: Evaluating Frontier and Open-Weight Models on Crypto-Asset Accounting Tasks
Related stories
Hugging Face Trending Papers
Jul 29
APEX-Accounting
We introduce APEX-Accounting, a benchmark built by Mercor in partnership with Ramp, to assess whether frontier models can do the real work of accountants. Tasks include reconciling accounts, accruing expenses, posting transactions, and producing reports.
arXiv AI
Jul 3
Meta-Benchmarks for Financial-Services LLM Evaluation
arXiv:2607. 01740v1 Announce Type: new Abstract: Public LLM leaderboards optimise for global average performance and do not capture the specific cognitive demands of financial-services work: a model that leads on MMLU-Pro may underperform on document-grounded compliance reasoning, and a coding leader may handle multi-turn customer interactions poorly.
arXiv AI
Aug 20
FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems
FinRCA-Bench is a deterministic synthetic benchmark comprising 2,250 accounts‑payable‑to‑bank reconciliation cases that span 14 operational tables and include 1,500 injected failures across 15 causal categories. The benchmark hides root‑cause labels and record‑level evidence contracts from models, enabling independent evaluation of evidence retrieval versus reasoning accuracy. Experiments show that retrieval architecture dramatically influences performance, with retrieval improvements raising macro‑required‑record recall from 0.83% to 77.70% and exact 16‑class accuracy from 2.05% to 72.44%.
arXiv AI
Jun 12
Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings
arXiv:2602. 07294v4 Announce Type: replace-cross Abstract: With the increasing deployment of Large Language Models (LLMs) in the finance domain, LLMs are increasingly expected to parse complex regulatory disclosures.
arXiv Computation and Language
Aug 27
FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review
FinRiskAtlas is a Chinese-language benchmark designed to evaluate large language models (LLMs) for financial risk review by focusing on decision‑aligned tasks rather than generic financial knowledge. It contains 9,742 instances across 53 task families, including 42 domain‑knowledge families and 11 downstream review operations defined by explicit evaluation contracts. The extended FinRisk‑Ask framework replays 680 pre‑action states from 104 professional trajectories, withholding future evidence during inference to assess evidence‑state control and request targeting. Results across 33 model configurations show that operation‑level evaluation yields distinct rankings and that knowledge‑based shortlisting can incur significant regret, while frequent use of the Ask branch does not necessarily improve evidence acquisition, highlighting gaps in broad financial capability scores.
Hugging Face Trending Papers
Aug 19
FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems
FinRCA-Bench is a synthetic benchmark designed to evaluate evidence retrieval and reasoning in financial AI systems, specifically for accounts‑payable‑to‑bank reconciliation. It contains 2,250 cases across 14 operational tables, with 1,500 injected failures in 15 causal categories and 750 hard‑negative cases, and hides root‑cause labels and evidence contracts to isolate retrieval performance. Experiments show that retrieval architecture dramatically affects accuracy, with structured retrieval methods like Typed Provenance Graph Retrieval vastly improving macro‑recall and exact‑class accuracy compared to dense semantic retrieval or classical ML.
arXiv Computation and Language
Sep 23
FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability
arXiv:2609.25192v1 Announce Type: new Abstract: Financial search is a highly demanding task for LLM agents, requiring not only a correct final answer but also temporally valid information retrieval,...
arXiv AI
Jun 3
BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents
arXiv:2606. 03829v1 Announce Type: new Abstract: Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed.
arXiv AI
Aug 7
FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
arXiv:2608. 06144v1 Announce Type: new Abstract: Most agent benchmarks evaluate tasks independently and cannot measure whether experience from one task helps with later tasks.
arXiv AI
Aug 6
FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables
arXiv:2608. 04077v1 Announce Type: new Abstract: Evaluating financial AI agents requires criteria aligned with real professional work.
arXiv Machine Learning
Aug 18
Mint-Agent: Introducing Finance-Native Agentic Foundation Models
arXiv:2608. 16386v1 Announce Type: cross Abstract: Financial agents must do more than recall domain knowledge: they must be both reliable, executing precise operations over grounded evidence, and executive, sustaining long-horizon research whose conclusions remain auditable.