arXiv:2608. 09019v1 Announce Type: cross Abstract: As generative AI increasingly becomes a common source of daily decision-making, including financial choices, it is critical to understand how people evaluate AI-generated financial advice.
By Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv Saha
arXiv:2609.24644v1 Announce Type: cross
Abstract: As people turn to generative AI for financial advice, these systems can personalize how they communicate and what they say. Whether these forms of pe...
By Hasibur Rahman, Benjamin R. Cowan, Smit Desai
arXiv:2606. 19887v1 Announce Type: cross Abstract: Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks.
By Chaeyun Kim, Daeyoung Park, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim
FinRED is an expert‑guided framework for generating and evaluating financial LLM red‑teaming tests. It maps global financial standards to specific threats, converts real documents into context‑rich prompts, and offers a validated rubric that reduces false negatives. The framework is deployed in South Korea’s Financial Security Institute sandbox and is gated for qualified researchers.
By Daeyoung Park, Chaeyun Kim, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim
The study examines how large language models (ChatGPT, Gemini, and Grok) embed religious bias in AI‑generated financial advice. Using 432 simulated advisor‑client interactions across four religious identities and three financial decisions, the authors find that only 12‑18% of advice is unbiased, with Gemini showing the most bias and ChatGPT comparable to Grok. The research identifies structural biases in model design and discursive mechanisms—such as religious anchoring and tone modulation—that vary by scenario, revealing a tension between personalization and neutrality in AI advisory services.
By Muhammad Salar Khan, Hamza Umer, Hasan Mahmud, Sandra Rothenberg
arXiv:2608. 02100v1 Announce Type: cross Abstract: As AI increasingly participates in human decision making, understanding how decision-making authority is distributed between humans and AI has become a fundamental behavioural question.
By Iman Munire Bilal, Yingcan Carol Wang, Ajan Raj, Filippo Giovagnini, Pranav Tewari, Yuwei Zhang, Mei-Chen Zoe Liou, Qamar Zaman
AgenticFinBench is a new benchmark comprising 230 real‑world financial tasks across eight categories, two languages, and four complexity levels, designed to capture the breadth and open‑ended nature of financial AI work. It introduces a reference‑free evaluation framework that uses question‑specific rubrics and a two‑stage claim audit to detect hallucinations, aligning assessment more closely with real‑world financial practice. Experiments on 18 agent configurations show that ignoring hallucinations inflates performance, that harness differences drive evidence grounding more than model differences, and that task complexity reduces quality scores, highlighting challenges in explaining financial impacts and comparing data consistently.
By Shu Zhou, Ziyang He, Yuxin Rao, Yudong Fang, Jiali Qian, Dongyuan Chen, Juntao Zhang, Yong Hu
The paper introduces SME-C, a human‑AI workflow that combines lightweight expert critiques with LLM‑driven refinement to create high‑quality, domain‑specific evaluation datasets and rubrics. In a financial business brief domain, SME‑C gathered 3,853 critiques from 411 professionals and used two LLM refinement strategies—one guided by written critiques and another by preference votes—to improve response quality. Blind validation with 100 additional professionals showed both strategies significantly outperformed the experts’ top‑ranked baseline, with critique‑based refinement yielding more domain‑grounded improvements and vote‑based refinement better at removing irrelevant content.
By Honson Ling, Kidus Admassu, Stephanie Rosenthal
FinAutoRubric is a system that lets experts define reusable evaluation guidance for financial research agents, which the agents then use to automatically generate, review, and validate task‑specific rubrics. The approach replaces costly fixed rubrics with query‑specific ones that incorporate expert standards and can be updated as new information becomes available. On three expert‑authored finance benchmarks, the automatically generated rubrics closely match human scoring and are preferred by in‑house analysts in blind reviews.
By Hoyoung Lee, Suyeol Yun, Jack Haverty, Yunju Cho, Meesong Kim, Daekyung Park, Sumin Kim, Jihoon Kwon, Jasmine Jia Geng, Andrew Chin, Yin Luo, Edward Tong, Yu Yu, Zach Golkhou, Minkyu Kim, Igor Halperin, Young Cha, Alejandro Lopez-Lira, Chanyeol Choi, Yongjae Lee
The paper introduces a benchmark for evaluating large language models on trustworthy analysis of earnings call transcripts. It proposes a numeric evidence evaluation method that assesses groundedness without expert annotation, and presents an automated pipeline that builds the ECTs-100 dataset from the top 100 S&P 500 constituents. The study also explores the failure mode of conscious incompetence, where models must recognize insufficient evidence and avoid hallucinations, finding that while groundedness is strong, correctness remains a challenge.
By Yingzhu Zhao, Vlad Pandelea, Han Yuan, Bo Hu, Wuqiong Luo, Li Zhang, Zheng Ma
arXiv:2607. 04103v3 Announce Type: replace-cross Abstract: Generative artificial intelligence is moving from general-purpose experimentation toward specialized applications across banking, capital markets, insurance, payments, and wealth management.
By Dennis Mao, Alessandra Lin, Yixin Kang, Yiqing Wang
The paper proposes a three‑engine architecture for reliable financial AI, combining a financial knowledge graph, a deterministic optimization engine, and a multi‑agent LLM layer. A coordination layer called context engineering selects and validates context from ERP systems, tools, and tenants to ensure numerical accuracy, traceability, and auditability. Observations from 15 deployments show that graph‑backed multi‑agent validation can reduce hallucination rates from 7‑9% to below 1%, while reusable mappings and validation policies cut deployment time from six to two weeks.