arXiv Machine Learning

When Does "Post-Call" Begin? An Availability Contract for Earnings-Call NLP Benchmarks

The paper introduces an Availability Contract to address timing mismatches in financial NLP benchmarks, requiring explicit declaration of input boundaries, evidence of availability, outcome start, and supported claims. Applying this framework to 552 earnings calls, the authors show that shifting from a call‑start proxy to a completion proxy changes return signs in 9.4% of calls and alters quintile rankings in 28.3%. In a volatility benchmark, moving target windows forward by one session increases mean squared error by 18.1‑21.4% while leaving point rankings unchanged, illustrating that labels and losses can be sensitive to the declared outcome clock.

arXiv Machine Learning
Aug 19

Temporal Leakage in Financial News NLP: A Multi-Architecture Audit with a Regime-Specific M&A Signal

The paper audits the impact of temporal leakage on financial-news direction prediction across 49,799 articles and 16 feature-model combinations, including TF‑IDF, MiniLM, FinBERT, and fine‑tuned RoBERTa‑large / DeBERTa‑v3‑large, as well as zero/few‑shot and LoRA probes of Llama‑3 and Qwen2.5. Random train‑test splits inflate MCC scores by 1.1× to 6.5×, with larger models and richer features showing greater gains, while end‑to‑end FinBERT fine‑tuning actually increases the gap. Only the mergers and acquisitions (M&A) category shows a positive locked‑test signal under near‑temporal chronological evaluation, with the signal localized to 2024‑2025 European‑tilted M&A semantics and not transferring to a 2009‑2020 U.S. corpus.

By Chenhao Xue, Raslen Guesmi, Siwei Feng, Yucheng Gong, Jacob Xavier Sundram, Jordan Pang, Lan Wang, Julian Kaljuvee
arXiv AI
Oct 16

Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts

The study investigates which inputs influence a language model’s financial forecasts by creating synthetic earnings announcements that independently reverse 20‑day momentum, commentary tone, and EPS surprise. Six instruction/chat checkpoint conversions from four model families (0.49‑7.61 B parameters) were tested with 2,016 prompts each, covering 384 primary scenarios. Results show that the contrast in primary EPS ranges from –0.19 to 86.80 percentage points, with signed tone and momentum contrasts often exceeding EPS effects; numerical diagnostic accuracy varied from 51.0 % to 95.8 %. The experiment measures observable input reliance and numerical recognition but does not establish internal reasoning mechanisms or return predictability.

By Alina Khaybullina
arXiv Computation and Language
Sep 4

DisclosureBeta: A Measurement-Channel Theory for Regime-Conditioned Betas from LLM-Read Risk Disclosures

The paper introduces DisclosureBeta, a theory that treats large language models (LLMs) as noisy measurement channels for a firm’s latent risk characteristics, integrating this noise into the asset‑pricing error budget. It establishes identification and consistency of regime‑conditional beta loadings within a piecewise‑stationary Fama‑French five‑factor framework, provides matching lower bounds, and proposes an adaptive estimator that blends text‑based and rolling‑window approaches, improving precision when price histories are short or regime‑breaks occur. The work also outlines a pre‑registered empirical evaluation on firms with thin price histories.

By Ping Kuen Wong
arXiv Machine Learning
Sep 24

Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel

The paper demonstrates that the outcome of a forecasting leaderboard is largely determined by the evaluator’s design choices rather than the models themselves. By fixing the data, horizon, and period, the authors varied three key evaluation decisions—unit of analysis, error pooling, and scoring metric—and showed that each can reverse or eliminate the apparent superiority of any forecasting method. The study also evaluates the practical impact of these choices on a deployed system, revealing that the selection rule captures a significant portion of the potential performance gain, and confirms the findings on an external public dataset.

By Md Rezwanul Islam, Wael Mohammed
arXiv Machine Learning
Oct 12

DataCupid: Stable Matching Repair for Training Data Acquisition with Delayed Feedback

The paper introduces RLR, a stable matching repair method that reuses valid observations while respecting matching capacity and handling delayed, out‑of‑order evaluations. RLR combines sound outcome certificates with batched matching schedules to manage unequal sample deficits, achieving a sharp issue‑round bound and separate delay and evidence‑lifetime bounds. The authors also present RLR‑P, a variant that monitors issue‑order prefixes without altering the sampling plan, and demonstrate through simulations that it reduces mean capped update‑certification time compared to a width‑based baseline, albeit with higher oracle CPU cost.

By He Sun, Junyuan Mao, Wenhao Jiang, Mingjun Xiao, Shinan Liu
arXiv AI
Sep 7

FinalityBench: An Effect-Level Benchmark for Agent Decisions Under Delayed and Conflicting Financial Finality

FinalityBench is an executable benchmark that tests how agents decide on shipping, re‑capturing, refunding, or waiting when a merchant’s payment processor, ledger, ERP, and bank feed receive delayed, duplicated, dropped, or reordered messages, causing contradictory beliefs about an order. The benchmark uses a hidden canonical event log and faulted delivery streams to generate system views, scoring each episode by the merchant’s terminal economic position relative to a privileged reference. It contains 321 tasks, including 45 twin pairs where all four views are identical yet the correct disposition differs, and evaluates nine programmatic policies, revealing that a ship‑on‑first‑sign policy performs best by accuracy but worst by paired loss, while a runtime‑gated irreversible‑action policy achieves 85.4% accuracy without losing money.

By Abhishek Sharma