The study examines whether financial sentiment tools that are validated against human labels also reliably predict market outcomes. Using a large corpus of securities class action messages linked to abnormal stock returns, the authors compare five sentiment instruments—VADER, Loughran‑McDonald, FinBERT, Twitter‑RoBERTa, and an LLM annotator—within a single pipeline. Results show that the alignment between human agreement and sentiment scores varies with sampling strategy and time horizon: conventional sampling favors same‑day associations, while fixed‑n panels yield similar correlations for both same‑day and one‑day‑ahead predictions, yet overall predictive rankings remain weak.
By AS Aravinthkakshan, Laven Srivastava, Harsh Nandwani
arXiv:2604. 27374v2 Announce Type: replace Abstract: As LLMs become credible readers of earnings calls, investor-relations Q\&A, guidance, and disclosure language, supervised financial NLP benchmarks increasingly function as decision evidence for model selection and deployment.
By Sidi Chang, Peiying Zhu, Yuxiao Chen, Rongdong Chai
arXiv:2607. 14174v1 Announce Type: new Abstract: Financial sentiment extraction has largely relied on news text and supervised extraction against return labels alone, leaving 10-K filings -- and volatility, the target risk disclosure is arguably best suited to informing -- comparatively unexplored.
By Sanggyu Sean Choi
arXiv:2606. 02255v1 Announce Type: cross Abstract: Human annotation is the empirical foundation of much NLP research, from dataset construction to model evaluation, but papers often leave unclear who produced the annotations and how the annotation process was controlled.
By Maria Kunilovskaya, Gagan Bhatia, Lisa Sophie Albertelli, Yanran Chen, Christian Greisinger, Lotta Kiefer, Christoph Leiter, Subhadeep Roy, Tewodros Achamaleh, Muhammad Arslan Manzoor, Sebastian Pohl, Yufang Hou, Steffen Eger
The paper presents the CDSP (context-conditional deliberation signal pipeline), which transforms investment committee meeting transcripts into structured predictive features. CDSP segments transcripts into topical chunks, assigns asset‑class context labels via a large language model, maps financial keywords to a taxonomy, and adds sentiment polarity and mention frequency features. Using these engineered features on 48 monthly meetings, the best model—combining sentence embeddings with CDSP features—achieves 73% accuracy and a 0.73 F1 score, outperforming a simple stock‑choice baseline, though the improvement is not statistically significant.
By Vivek Batra, Kristin Chen, Sanjiv Das, Samuel Judge, Harshad Khadilkar, Sukrit Mittal, Amir Nasrollahzadeh, Daniel Ostrov, Jacob Sisk
arXiv:2608. 04200v1 Announce Type: cross Abstract: Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically useful return predictability.
By Fusheng Luo
The paper evaluates twelve financial sentiment models—including dictionary-based methods, finance-specific transformers, and open-source large language models—using linguistic and economic validity criteria. General-purpose LLMs match finance-specific transformers in classification performance but do not yield stronger economic relationships. While several models correlate with earnings surprises, none shows a significant link to next‑day stock returns, and performance is strongest for large earnings beats or misses.
By Arslan Bisharat, Oudom Hean
arXiv:2609.36194v1 Announce Type: new
Abstract: Extracted sentiment directions can vary across samples even when downstream sentiment classification remains accurate. To evaluate direction reproducib...
By Muhammad Abdullahi Said, Abass Oguntade, Elisha Komolafe, Babangida Sani, Fatima Muhammad Adam, Muhammad Sammani Sani
arXiv:2605. 15416v2 Announce Type: replace-cross Abstract: Jung et al.
By Gaojie Jin, Yong Tao, Lijia Yu, Tianjin Huang
The paper investigates how sentence‑specificity scores can guide the selection of revisions in collaborative technical documentation. It compares two predictors—SpeciTeller and a target‑adapted model by Ko et al.—across Wikipedia and three technical corpora, finding that the predictors rank sentences differently and that SpeciTeller can improve direction‑valid selection rates in certain datasets. The study also shows that filtering and token‑length adjustments alter but do not reconcile these ranking differences.
By Rocker D'Antonio, Thomas Benton Townsend, Dimitrios Michael Manias
This scoping review examines 421 studies (2015‑2026) on natural language processing applied to student evaluation of teaching comments. It maps the technical evolution from lexicons and classifiers to transformers and large language models, and evaluates four value dimensions. The review identifies a significant gap between actionable outputs (61.3%) and intended‑user evaluation (11.6%), highlighting limited progress in educational value and robustness.
By Jeff Eicher, Rafael da Silva
Large Language Model judges are commonly used to rank texts via pairwise comparison, with reliability traditionally measured by position bias, transitivity, and pairwise agreement. This paper argues that these proxies are misleading because they are dominated by close‑rank‑gap pairs, which contribute little to the overall ranking, while far‑gap pairs carry the true ranking signal. Experiments on simulations and human‑rated corpora show weak correlation between the proxies and actual ranking accuracy, suggesting judges should be evaluated using rank‑gap‑conditional metrics against human rankings.
By Bruno Brocai, Maria Becker