RupeeBias is a new benchmark that audits demographic bias in large language models (LLMs) when they provide economic guidance in India. It contains 39,150 prompts across four use cases—salary estimation, salary increment estimation, counter‑offer recommendation, and service pricing recommendation—varying 87 India‑specific demographic identifiers such as caste, religion, regional identity, gender, disability, and urban‑rural location. Evaluations of nine LLMs show that outputs differ by an average of 20.2% when only the demographic identifier changes, revealing systematic disparities across all six axes.
By Pavithra P M Nair, Bhavik Talaviya, Shourya Bhushan, Rahul Pankajakshan, Seema Guruvadoo, Avinash Agarwal, Gilad Gressel, Krishnashree Achuthan
arXiv:2607. 28934v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender.
By Martin Lukk (University of Toronto)
The study investigates how large language models (LLMs) compensate for missing financial information by substituting user identity cues. Using 96,600 prompts to Llama‑3.1‑8B‑Instruct, the authors varied the amount of financial facts provided while keeping the underlying finances constant, and measured changes in recommended equity allocations across 100 financial profiles, 138 personas, and seven disclosure conditions. Results show that as financial facts are removed, the influence of identity on advice grows dramatically—from 5 % of variation with full disclosure to 96 % with none—while household size becomes the most reliable predictor when evidence is scarce, and gender effects persist even after controlling for standard errors.
"whyItMatters":"The findings highlight that LLM‑based advisory systems can produce biased financial recommendations when users provide incomplete information, underscoring the need for audits that reflect real‑world disclosure levels and consider the full spectrum of user identities."
By Saanvi Khetan, Sankar Balasubramanian
The study examines how large language models (ChatGPT, Gemini, and Grok) embed religious bias in AI‑generated financial advice. Using 432 simulated advisor‑client interactions across four religious identities and three financial decisions, the authors find that only 12‑18% of advice is unbiased, with Gemini showing the most bias and ChatGPT comparable to Grok. The research identifies structural biases in model design and discursive mechanisms—such as religious anchoring and tone modulation—that vary by scenario, revealing a tension between personalization and neutrality in AI advisory services.
By Muhammad Salar Khan, Hamza Umer, Hasan Mahmud, Sandra Rothenberg
arXiv:2608. 09019v1 Announce Type: cross Abstract: As generative AI increasingly becomes a common source of daily decision-making, including financial choices, it is critical to understand how people evaluate AI-generated financial advice.
By Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv Saha
The study investigates how people evaluate AI-generated financial advice by conducting a randomized vignette experiment with 285 U.S. adults. Participants were presented with consistent financial recommendations delivered in three styles—AI, expert, and online community—alongside source labels. The results show that advice style most strongly influenced message and safety appraisals, expert labels increased perceived source knowledge, and decision context shaped risk and safety judgments, with these appraisals explaining a large portion of overall quality, trust, and intended reliance.
By Aryan Ramchandra Kapadia, Eshwar Chandrasekharan, Koustuv Saha
FinRED is an expert‑guided framework for generating and evaluating financial LLM red‑teaming tests. It maps global financial standards to specific threats, converts real documents into context‑rich prompts, and offers a validated rubric that reduces false negatives. The framework is deployed in South Korea’s Financial Security Institute sandbox and is gated for qualified researchers.
By Daeyoung Park, Chaeyun Kim, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim
arXiv:2606. 19887v1 Announce Type: cross Abstract: Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks.
By Chaeyun Kim, Daeyoung Park, Junghwan Kim, Jinyoung Jeong, Eunji Song, Yongtaek Lim, Minwoo Kim
The study evaluates ten bias audit instruments across ten advanced language models on occupational gender, age, and socioeconomic status. While each tool reliably detects bias, their rankings of model performance are essentially random, indicating that different audits measure distinct constructs. The findings show that a single audit can identify bias direction within its own framework, but no audit can consistently rank models against one another.
By William Guey, Pierrick Bougault, Wei Zhang, Vitor D. de Moura, Jos\'e O. Gomes
Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways. Detecting a conflict is only the first step: review workflows may also need to determine its type, since numerical, temporal, referential, factual, and normative inconsistencies require different evidence and downstream checks.
arXiv:2607. 26368v1 Announce Type: cross Abstract: Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways.
By Aman Kumar, Lasitha Vidyaratne, Dipanjan D Ghosh, Arnab Chakrabarti, Ahmed K Farahat
Investor's Last Exam (ILE) is a new equity‑research benchmark created by experienced buy‑side investors and fund managers. It comprises a versioned financial database of 3.43 million document versions, a task taxonomy aligned with professional analyst deliverables, and an expert‑led framework that links scoring criteria to sources, calculations, and assumptions via financial dependency graphs. The benchmark evaluates 86 assignments, jointly measuring research quality, forecast accuracy, and simulated investment performance, and finds that even top models like GPT‑6 Astra and Claude Opus 5.5 struggle to translate research capability into predictive performance.
By Huangfei Jiang, Anning Hu, Kai Sheng, Yuchen Cai, Rui Ge, Xiangyu Kong, Jianghao Lin, Shuzi Chen, Jiyi Weng, Junchi Yan