The paper explores using LLM-generated annotations to train smaller BERT models for literary sentiment analysis. Five BERT variants were fine‑tuned with ordinal regression on 10,889 sentences labeled by Gemma 4 31B on a ten‑point scale, and evaluated on 1,923 human‑annotated sentences from Hemingway’s *The Old Man and the Sea*. BERT‑large achieved a Kendall’s τ of 0.589, close to the teacher’s 0.604, and BERT‑base outperformed the teacher on non‑neutral sentences with an MAE of 0.726 versus 1.116.
arXiv:2606. 29614v1 Announce Type: cross Abstract: This study examines whether supervised fine-tuning remains necessary for Turkish sentiment analysis in the era of large language models.
The paper investigates how perspectivist NLP models perform when predicting annotations from different annotators, focusing on those who diverge from the majority or provide fewer annotations. It compiles the largest set of perspectivist datasets with ordinal labels and evaluates various algorithms across them. The study finds that models tend to perform better on annotators who agree with others, while the rate at which annotators provide responses does not significantly affect performance.
arXiv:2606. 18922v1 Announce Type: cross Abstract: Figurative language and negation are two areas that challenge current language models, however, both are widely used throughout written and spoken language.
The paper investigates how data augmentation can improve sentiment classification on small datasets. Three augmentation techniques—Easy Data Augmentation (EDA), back‑translation, and contextual token substitution—are evaluated with both traditional classifiers (Logistic Regression, Random Forest) and BERT. Results show that all methods boost performance, with contextual augmentation best for BERT and EDA/back‑translation benefiting traditional models.
Institutions collect far more open-ended teaching-evaluation feedback than they read. A prior study introduced a validated protocol for classifying such comments by thematic category and sentiment, built from a documented annotation guide, an intra-annotator reliability measurement, stratified cross-validation, and a held-out evaluation on a Spanish institutional corpus with a frozen-encoder design.
arXiv:2609.36194v1 Announce Type: new Abstract: Extracted sentiment directions can vary across samples even when downstream sentiment classification remains accurate. To evaluate direction reproducib...
The paper introduces KESA, a knowledge‑enhanced approach for sentence‑level sentiment analysis that incorporates sentiment knowledge through two auxiliary tasks: sentiment word cloze and conditional sentiment prediction. These tasks use prior sentiment polarity to guide the selection of sentiment words and the prediction of overall sentiment, respectively, and explore label combination methods to unify multiple label types. Experiments show that KESA consistently outperforms pre‑trained models and complements existing knowledge‑enhanced post‑training methods.
The paper introduces CS‑SeLeRoSa, a synthetic dataset of over 600k code‑mixed Romanian sentences with English, German, French, and Italian, generated via four methods (random replacement, phrase translation, dependency parsing, and large language models). It evaluates multilingual large language models on satire detection in this dataset and a human‑annotated gold benchmark, CS‑SeLeRoSa‑Gold, showing that newer models like Gemma 4 31B outperform older open models and achieve competitive zero‑shot performance, with fine‑tuning further improving results.
The paper introduces BASSE, a multilingual meta‑evaluation dataset containing 2,040 human‑rated abstractive summaries produced manually or by five LLMs with four prompts. Annotators scored each summary on coherence, consistency, fluency, relevance, and 5W1H using a 5‑point Likert scale. Benchmarking shows proprietary LLM‑judge models best align with human judgments, followed by criteria‑specific automatic metrics, while open‑source judge LLMs perform poorly.
IDEAlign introduces a new protocol for evaluating the similarity of large language model (LLM) annotations to expert judgments. It uses pick‑the‑odd‑one‑out tasks to capture expert similarity and benchmarks various similarity methods—including text embeddings, topic models, and LLM-as-a-judge—against these human ratings. Applied to educational datasets, the study finds that most metrics miss nuanced expert dimensions, with LLM-as-a-judge performing best yet still insufficient for full expert alignment.
The study evaluates how well three sentiment‑analysis tools (TextBlob, VADER, Twitter‑roBERTa‑base) and three large language models (Qwen3‑32B, GPT‑OSS‑120B, Llama‑4‑Maverick‑17B) agree with six human raters on 100 tweets. Agreement was measured with Cohen’s and Fleiss’ kappa, revealing only fair inter‑human agreement and higher concordance for binary sentiment labels than for three‑class labels. Twitter‑roBERTa‑base achieved the strongest alignment with humans, especially for negative versus non‑negative sentiment, while the LLMs showed substantial agreement among themselves and moderate to substantial alignment with humans, particularly for positive versus non‑positive classifications. The findings emphasize that domain‑specific fine‑tuning and human‑centered evaluation are essential for reliable social media sentiment analysis.
arXiv:2607. 11873v1 Announce Type: cross Abstract: Institutions collect far more open-ended teaching-evaluation feedback than they read.