Terminal X’s Eval Agent evaluates GPT‑6.1’s outputs using a 54‑criterion rubric, achieving an 88.6% match with senior investor judgment. The system detects 25% more deviations from a fund’s mandate compared to previous methods.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at OpenAI Blog.
The paper evaluates an agentic harness that combines a 27B language model, financial calculations, narrative templates, and validation checks to produce corporate and investment banking presentation decks. Using a panel of five judges, the system consistently scores higher than a baseline model that generates directly from a short prompt, with scores ranging from 20.4 to 33.6 out of 95. However, judge variability and changes in grading criteria make it challenging to discern small improvements in deck quality.
arXiv:2610. 32380v1 Announce Type: new Abstract: Large language models (LLMs) can read startup pitch decks, but letting them emit investment scores directly invites score inflation, run-to-run variance and rationales that are hard to audit.
FIDES-Live extends the FIDES framework to audit sequential trading agents by separating decision validity, numerical commitment adherence, execution and ledger reconciliation, and receipt‑grounded reporting. In a retrospective audit of 7,299 cryptocurrency account‑rounds and a 310‑round equity pilot, decision validity reached 97.35% and structured reporting accuracy 90.81%, while commitment adherence overall was 94.36% but dropped to 62.63% for non‑empty order proposals. The study shows that all cash and position transitions reconcile with recorded fills, and highlights the need to report concordance, evidence coverage, activity, and financial outcomes separately.
Sequential trading agents produce decisions, orders, execution receipts, account states, and reports that need not agree, even when the final portfolio value is correctly computed. We present FIDES-Li...
arXiv:2607. 24889v1 Announce Type: cross Abstract: Financial models combine public disclosures with analyst assumptions to produce forecasts and valuations.
VERITYGATE is a four‑gate framework that checks whether LLM‑generated narrations adhere to a fixed schema of declared evidence IDs, entities, numbers, and claim types, rather than verifying every fact in the prose. In experiments with GPT‑4o‑mini, Llama‑3.3‑70B, and Claude Sonnet 4.6 on 900 grounded‑ungrounded pairs, the framework identified high failure rates (up to 80.3% for mini claims) and demonstrated that a single repair pass can improve claim survival rates. The authors also provide code, data, and preliminary human studies to validate the rules and highlight gaps between schema compliance and correct prose. "whyItMatters":"The framework offers a systematic way to evaluate and improve the faithfulness of LLM explanations to structured evidence, revealing significant failure rates and the impact of repair strategies."