Hugging Face Blog

Comparing the Performance of LLMs: A Deep Dive into Roberta, Llama 2, and Mistral for Disaster Tweets Analysis with Lora

arXiv Computation and Language
Sep 16

Can LLMs Follow the Pulse of a Crisis? Evaluating Crisis Sentiment in Bangladesh's July Uprising

arXiv:2609.16997v1 Announce Type: new Abstract: Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapid...

By Md. Samiul Alim, Mahir Shahriar Tamim, Tanvir Ahmed Khan, Sharjil Khan, Rafia Ferdous Duti, Shahriyar Zaman Ridoy, Mohammad Ali Moni
arXiv Computation and Language
Oct 8

Nobody Truly Agrees on Sentiment: Humans, Bespoke Tools, and LLMs Struggle with Social Media Texts

The study evaluates how well three sentiment‑analysis tools (TextBlob, VADER, Twitter‑roBERTa‑base) and three large language models (Qwen3‑32B, GPT‑OSS‑120B, Llama‑4‑Maverick‑17B) agree with six human raters on 100 tweets. Agreement was measured with Cohen’s and Fleiss’ kappa, revealing only fair inter‑human agreement and higher concordance for binary sentiment labels than for three‑class labels. Twitter‑roBERTa‑base achieved the strongest alignment with humans, especially for negative versus non‑negative sentiment, while the LLMs showed substantial agreement among themselves and moderate to substantial alignment with humans, particularly for positive versus non‑positive classifications. The findings emphasize that domain‑specific fine‑tuning and human‑centered evaluation are essential for reliable social media sentiment analysis.

By Himarsha R. Jayanetti, Sivakanesan Dhanushkanda, Shuai Hao, Michael L. Nelson, Michele C. Weigle
arXiv AI
Aug 7

Beyond Sentiment: Comparing Traditional NLP and LLM-Based Multi-Dimensional Analysis for Political News Evaluation

arXiv:2608. 05155v1 Announce Type: cross Abstract: Traditional sentiment analysis (SA) models, while effective for polarity classification, provide limited insight into the rhetorical, ideological, and framing dimensions of political discourse -- dimensions that are central to research in the social sciences and humanities (SSH).

By Maryam Fooladi, Federico Bottino
arXiv AI
Aug 7

Automatic Detection of Deaths from Social Networking Sites

arXiv:2608. 05183v1 Announce Type: cross Abstract: This dissertation analysed and discussed the differences in linguistic characteristics between pre-mortem and post-mortem social media content, and reported machine learning (ML) classifiers that achieved high performance in automatically detecting deaths of social networking site users from posts associated with their profiles.

By Nuhu Ibrahim, Riza Batista-Navarro
arXiv Computation and Language
Sep 11

Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection

The paper explores using a large language model (LLM) to generate background context for social media posts and tests four ways to integrate this context into a Sentence-BERT (SBERT) hate‑speech detection classifier. The methods include text concatenation, embedding concatenation, hierarchical transformer fusion, and LLM‑driven text enhancement. Experiments on the Latent Hatred dataset of implicitly hateful tweets and the MAMI dataset of misogynous memes show that adding generated context can raise F1 scores by up to 3 points in textual and 6 points in multimodal settings compared to a zero‑context baseline, with embedding concatenation yielding the best results.

By Joshua Wolfe Brook, Ilia Markov
arXiv Computation and Language
Oct 28

Where Does Rhetorical Distortion Live? A Multi-Platform, Multi-Language Measurement of Influencer Content

The study introduces a unified method for measuring rhetorical distortion across six social media platforms and two languages, distinguishing between creator posts and their replies. It identifies five distortion dimensions—Significance Inflation, Anxiety Manufacturing, Novelty Claims, Loaded Language, and Temporal Distortion—using a three‑tier pipeline that combines lexical rules, negative filters, and LLM verification. Results show that distortion is far more strongly linked to the platform than content type, with X/Twitter exhibiting the highest distortion scores, and that the audience layer (replies) carries significantly more distortion than creator posts.

By Xucheng Yu, Jiangping Chen, Haohan Wang