AStockArena is a real‑time decision‑auditing benchmark for continuous trading in China’s A‑share market, evaluating 13 open‑ and closed‑source models over 3,198 sequential decisions. The benchmark reconstructs actual account actions by aligning reports, trade requests, execution responses, and account states, then audits them with eight checks covering execution traceability, evidence sufficiency, and financial meta‑awareness. Across 1,070 trading points, 72.99% show at least one audit gap, with evidence gaps most common, and historical replay demonstrates that flagged trades significantly affect portfolio outcomes.
By Zeyu Liu, Sing Kwong Lai, Xuanzhi Feng, HaoYang Shang, Yang Li, Zihao Fang, Xuan Liu
The paper introduces PACT, a benchmark designed to evaluate how well enterprise AI assistants follow compliance rules when faced with various pressures such as persistent users or hurried managers. PACT covers twelve regulated domains and forty-eight realistic multi‑turn scenarios, pairing each rule with a shortcut that violates it and applying different pressures across wording and system‑prompt modes. Using PACT, the authors profile six metrics of compliance and aggregate them into a PACTScore, revealing significant variability among 22 LLM models and that even top performers misapply rules 6–10% of the time, with user pressure increasing violations by 65% on average.
By Mika Okamoto, Ansel Kaplan Erol
The paper introduces a compliance-first AI architecture for regulated finance, treating regulation as an orientation layer rather than a rigid rule set. It uses a regulatory intent matrix and a governed policy compiler to translate regulatory requirements into concrete prohibitions, obligations, and runtime budgets, while maintaining proportional committee activation and supervisory oversight. Evidence and decisions are recorded on a permissioned DAG with deterministic timestamps, enabling replay, provenance checks, and clear attribution of failures, and clause-level legal indexing ensures portability across the DACH region and the EU.
By Walter Kurz, Reinhard Magg
The paper investigates how procedural traces—detailed step-by-step accounts of a language model’s reasoning—affect the decision-making of LLM overseers tasked with auditing another model’s outputs. Using signal detection theory, the authors evaluated five overseers on 19 compliance tasks, finding that while error detection remains high when disconfirming evidence is always visible, more elaborate traces shift the decision criterion toward rejection, leading to increased false alarms. The study also shows that providing option labels reduces the stated inability to link evidence to options, yet some overseers still exhibit residual rejection of correct work that grows with trace detail.
By Zihan Chen, Di Zhu, Lei Zheng, Weiling Li
Sequential trading agents produce decisions, orders, execution receipts, account states, and reports that need not agree, even when the final portfolio value is correctly computed. We present FIDES-Li...
arXiv:2608. 16852v1 Announce Type: new Abstract: Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy.
By Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu, Pratinav Seth
The study examines how different harness configurations affect a fixed large language model’s compliance with banking conduct rules. Four open‑weight models were tested across five harnesses on 156 pre‑registered retail‑banking scenarios, yielding 3,120 two‑turn episodes scored for rule violations and over‑refusals. Results show that while overall compliance rates varied modestly (87.3%–92.1%), the impact of a harness differed markedly between models, with some models’ best and worst harnesses differing by over 10 percentage points and a same‑model compliance reviewer reducing violations from 9.6% to 6.4%.
whyItMatters":"The findings suggest that validating a model’s compliance requires testing the specific model‑harness pair rather than the model alone, highlighting the importance of action logs over conversation transcripts for accurate conduct assessment."
By Mehmet Murat Huyut
FIDES-Live extends the FIDES framework to audit sequential trading agents by separating decision validity, numerical commitment adherence, execution and ledger reconciliation, and receipt‑grounded reporting. In a retrospective audit of 7,299 cryptocurrency account‑rounds and a 310‑round equity pilot, decision validity reached 97.35% and structured reporting accuracy 90.81%, while commitment adherence overall was 94.36% but dropped to 62.63% for non‑empty order proposals. The study shows that all cash and position transitions reconcile with recorded fills, and highlights the need to report concordance, evidence coverage, activity, and financial outcomes separately.
By Arther Tian, Alex Ding, Simon Wu, Aaron Chan
arXiv:2608. 09025v1 Announce Type: new Abstract: Financial agents can turn correct context into an unauthorized effect: a customer-facing commitment, trade, or deployed policy.
By Rui Tang, Qiangqiang Liu, Yichi Zhang, Youwei Wang, Xi Chen, Chen Dong
The paper introduces FARSIGHT, a framework for evaluating the robustness and security of financial trading agents powered by large language models. It assesses agents on their resilience to market turbulence, such as flash crashes, and their vulnerability to three types of attacks: on information sources, on the agents themselves, and on agents acting as attackers. Applying FARSIGHT to 15 academic trading schemes reveals that 80% fail at least one robustness test and all exhibit security weaknesses, highlighting the risk of market-wide crashes from both accidental misjudgments and deliberate attacks.
By Mengxiao Wang, Nitesh Saxena
The paper investigates whether large language model (LLM) compliance systems truly base their verdicts on the regulatory rules they are given. By systematically deleting, swapping, or negating rules across five models and 20 regulatory domains, the authors find that verdicts and internal compliance representations change little, indicating that models often ignore the supplied rule. Even the guard model, adapted to a custom rule taxonomy, shows minimal rule sensitivity and only marginally better than chance.
The study highlights that accuracy alone does not guarantee that a compliance verdict is grounded in the supplied rule, underscoring the need for more robust auditing of regulatory rule sensitivity in LLM systems.
By Saisab Sadhu, Aadit Sengupta, Vinay kumar Sankarapu, Pratinav Seth
PRIME (Precommitted Rules for Inference from Minimal Evidence) is a framework designed to improve the reliability of large language model agents when verifying records against organizational requirements. It predefines evidence routes for each possible conclusion, records check results in a ledger, and refuses any conclusion that is not fully justified by the recorded evidence. In experiments on 480 public‑record tasks, PRIME eliminated unjustified conclusions entirely and increased the average score for justified conclusions from 30.1 to 59.2 out of 100.
By Hamed Khosravi, Xiaoming Huo