The Data Differentiator Framework (DDF) is a composite scoring method that evaluates candidate data assets by combining a mutual‑information redundancy penalty, a cost‑normalized marginal utility score from an ablation study, and explicit governance inputs such as trust and strategic weights. It is executed through the Strategic Data Sourcing Agent Ecosystem (SDSAE), which bypasses costly model retraining for assets that fail governance or redundancy checks. In a credit‑card fraud benchmark, only the genuinely differentiated asset passed the significance gate and received an acquisition recommendation, while a fully redundant asset scored lowest.
arXiv:2606. 09556v1 Announce Type: new Abstract: AI Scientist agents are often evaluated as if capability were mainly a function of model quality, prompting, or reasoning scaffolds.
FinRED is an expert‑guided framework for generating and evaluating financial LLM red‑teaming tests. It maps global financial standards to specific threats, converts real documents into context‑rich prompts, and offers a validated rubric that reduces false negatives. The framework is deployed in South Korea’s Financial Security Institute sandbox and is gated for qualified researchers.
arXiv:2606. 19887v1 Announce Type: cross Abstract: Existing safety benchmarks target general adversarial scenarios but miss finance-specific risks.
arXiv:2606. 29771v1 Announce Type: new Abstract: LLM agents are increasingly cast as autonomous portfolio managers, and benchmarks have moved from financial question-answering to sequential trading.
Agentic systems have widened the gap between producing candidate outputs and reviewing them. This paper asks a practical architectural question: should domain specialization be built into an evaluator's weights, or into the rule that decides when its judgment can be trusted?
arXiv:2607. 24889v1 Announce Type: cross Abstract: Financial models combine public disclosures with analyst assumptions to produce forecasts and valuations.
arXiv:2607. 26313v1 Announce Type: cross Abstract: Agentic systems act, so a defect in the evidence they retrieve becomes a wrong action with a currency cost.
The paper introduces FinSkillBench, a benchmark of 2,603 financial AI agent episodes across portfolio construction, risk management, and fundamental analysis, evaluated with deterministic verifiers. Experiments on 9 models and 3 resource settings show that curated skill packages improve performance by +16.2 points, while in‑episode skill generation adds only +0.5 points. Decomposing the premium reveals that executable tools contribute +19.5 points, documentation +5.6 points, and their combination is subadditive, with the effect varying by workflow type.
PRISM is a training‑free framework for financial information retrieval that combines prompt engineering, in‑context learning, and a candidate refinement module, with optional multi‑agent coordination. On FiQA‑2018 it improves Recall@100 from 0.7783 to 0.8326, and on FinAgentBench it achieves a Cohere‑refined top‑50 pool that raises NDCG@5 while cutting per‑query cost by about 35%. The authors also release an expert‑annotated evaluation set, a label‑robustness sweep, and a Pareto frontier of quality‑cost‑latency trade‑offs, and PRISM ranks third on the private FinAgentBench leaderboard without fine‑tuning.
arXiv:2609.15219v1 Announce Type: new Abstract: Founders face two linked decisions: whether to pursue an idea before founding, and which operating actions and capital partners fit afterward. We prese...
arXiv:2608. 16055v1 Announce Type: new Abstract: Existing agent benchmarks ask whether the agent finished the task.
arXiv:2607. 29252v1 Announce Type: cross Abstract: Reliable evaluation of open-ended LLM outputs requires fine-grained rubrics, yet expert curation is costly and difficult to scale.