← Back to all news
arXiv AI By Hoyoung Lee, Suyeol Yun, Jack Haverty, Yunju Cho, Meesong Kim, Daekyung Park, Sumin Kim, Jihoon Kwon, Jasmine Jia Geng, Andrew Chin, Yin Luo, Edward Tong, Yu Yu, Zach Golkhou, Minkyu Kim, Igor Halperin, Young Cha, Alejandro Lopez-Lira, Chanyeol Choi, Yongjae Lee

FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents

Read the original on arXiv AI →

FinAutoRubric is a system that lets experts define reusable evaluation guidance for financial research agents, which the agents then use to automatically generate, review, and validate task‑specific rubrics. The approach replaces costly fixed rubrics with query‑specific ones that incorporate expert standards and can be updated as new information becomes available. On three expert‑authored finance benchmarks, the automatically generated rubrics closely match human scoring and are preferred by in‑house analysts in blind reviews.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 12

Hindsight Structured Advice Distillation: Steering Executor Agents with Advisors Trained on Their Failures and Task Characteristics

The paper introduces Hindsight Structured Advice Distillation (HSAD), a method that trains a small advisor model to generate structured guidance for a frozen executor language‑model agent. By analyzing executor failures on a training set, a teacher writes advice that would have helped, and the advisor learns to produce this advice from input alone, adding minimal latency. Experiments on the FrontierFinance benchmark show that an 8B advisor boosts the qualification rate of Samaya’s research agent from 52.9% to 56.6%, surpassing the agent’s high‑effort configuration while cutting cost per query by 37%, and the approach also improves performance on WebShop, BIRD text‑to‑SQL, and ExpertLongBench.

By O. Ozan Koyluoglu
agentsfine-tuningefficiencybenchmarksdata-engineering
More like this →
arXiv AI
Aug 7

FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows

arXiv:2608. 06144v1 Announce Type: new Abstract: Most agent benchmarks evaluate tasks independently and cannot measure whether experience from one task helps with later tasks.

By Bo Deng (Beihang University, Qwen DianJin Team, Alibaba Cloud Computing), Kang Zhou (Qwen DianJin Team, Alibaba Cloud Computing), Lifan Guo (Qwen DianJin Team, Alibaba Cloud Computing), Chongyang Tao (Beihang University), Xuanren Chen (Beihang University), Chenggang Xie (Beihang University), Renzhao Liang (Beihang University), Feng Chen (Qwen DianJin Team, Alibaba Cloud Computing), Chi Zhang (Qwen DianJin Team, Alibaba Cloud Computing)
llmsagentsbenchmarks
More like this →