arXiv AI By Sidi Chang, Peiying Zhu, Yuxiao Chen, Rongdong Chai

Measurement Risk in Supervised Financial NLP: Rubric and Metric Sensitivity on JF-ICR

Read the original on arXiv AI →

arXiv:2604. 27374v2 Announce Type: replace Abstract: As LLMs become credible readers of earnings calls, investor-relations Q\&A, guidance, and disclosure language, supervised financial NLP benchmarks increasingly function as decision evidence for model selection and deployment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 11

Same Day, Same Story; One Day Ahead, a Different Signal: The Dual Validity of Financial Sentiment

The study examines whether financial sentiment tools that are validated against human labels also reliably predict market outcomes. Using a large corpus of securities class action messages linked to abnormal stock returns, the authors compare five sentiment instruments—VADER, Loughran‑McDonald, FinBERT, Twitter‑RoBERTa, and an LLM annotator—within a single pipeline. Results show that the alignment between human agreement and sentiment scores varies with sampling strategy and time horizon: conventional sampling favors same‑day associations, while fixed‑n panels yield similar correlations for both same‑day and one‑day‑ahead predictions, yet overall predictive rankings remain weak.

By AS Aravinthkakshan, Laven Srivastava, Harsh Nandwani
arXiv AI
Sep 25

Human Agreement and Return Association Are Not Interchangeable Criteria

The paper examines whether human agreement and return association can be used interchangeably as criteria for validating sentiment tools in financial NLP. Using a large corpus of securities class action messages linked to abnormal stock returns, the authors compare five sentiment instruments and find that the relationship between human agreement and predictive validity varies with sampling conventions and score representations. They conclude that benchmark agreement establishes semantic validity but does not guarantee predictive rankings, and that message volume in a spam‑heavy conversation does not predict market damage or settlement size.

By AS Aravinthakshan, Laven Srivastava, Harsh Nandwani
Hugging Face Trending Papers
Jul 29

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in qualitatively different ways. Detecting a conflict is only the first step: review workflows may also need to determine its type, since numerical, temporal, referential, factual, and normative inconsistencies require different evidence and downstream checks.

arXiv AI
Jun 12

Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings

arXiv:2602. 07294v4 Announce Type: replace-cross Abstract: With the increasing deployment of Large Language Models (LLMs) in the finance domain, LLMs are increasingly expected to parse complex regulatory disclosures.

By Yidong Jiang, Junrong Chen, Eftychia Makri, Jialin Chen, Peiwen Li, Ali Maatouk, Leandros Tassiulas, Eliot Brenner, Bing Xiang, Rex Ying