The Flow has not summarised this story yet — read it at arXiv Computation and Language.
FinRiskAtlas is a Chinese-language benchmark designed to evaluate large language models (LLMs) for financial risk review by focusing on decision‑aligned tasks rather than generic financial knowledge. It contains 9,742 instances across 53 task families, including 42 domain‑knowledge families and 11 downstream review operations defined by explicit evaluation contracts. The extended FinRisk‑Ask framework replays 680 pre‑action states from 104 professional trajectories, withholding future evidence during inference to assess evidence‑state control and request targeting. Results across 33 model configurations show that operation‑level evaluation yields distinct rankings and that knowledge‑based shortlisting can incur significant regret, while frequent use of the Ask branch does not necessarily improve evidence acquisition, highlighting gaps in broad financial capability scores.
arXiv:2608. 06108v1 Announce Type: new Abstract: Investment competence is inherently personalized: the same market evidence can justify different actions for investors with different goals, horizons, portfolios, and risk boundaries.
arXiv:2607. 18240v1 Announce Type: new Abstract: Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent.
arXiv:2608. 13706v1 Announce Type: cross Abstract: Existing defenses against hallucination in retrieval-augmented and multi-agent pipelines remain partial: evidence is trusted despite modality disagreement, debate verifies an aggregate report rather than individual claims, and such verification occurs only after drafting, leaving inter-agent errors undetected until the final text.
FinRCA-Bench is a synthetic benchmark designed to evaluate evidence retrieval and reasoning in financial AI systems, specifically for accounts‑payable‑to‑bank reconciliation. It contains 2,250 cases across 14 operational tables, with 1,500 injected failures in 15 causal categories and 750 hard‑negative cases, and hides root‑cause labels and evidence contracts to isolate retrieval performance. Experiments show that retrieval architecture dramatically affects accuracy, with structured retrieval methods like Typed Provenance Graph Retrieval vastly improving macro‑recall and exact‑class accuracy compared to dense semantic retrieval or classical ML.
FinRCA-Bench is a deterministic synthetic benchmark comprising 2,250 accounts‑payable‑to‑bank reconciliation cases that span 14 operational tables and include 1,500 injected failures across 15 causal categories. The benchmark hides root‑cause labels and record‑level evidence contracts from models, enabling independent evaluation of evidence retrieval versus reasoning accuracy. Experiments show that retrieval architecture dramatically influences performance, with retrieval improvements raising macro‑required‑record recall from 0.83% to 77.70% and exact 16‑class accuracy from 2.05% to 72.44%.