arXiv Machine Learning September 7, 2026 By E. Cho Smith, Samuel Ho, Dawn Laux

A Repeated-Measurement Study for Cultural Analytics of English Song Lyrics Using Five Large Language Models

Read the original on arXiv Machine Learning →

The paper evaluates five large language models as zero‑shot annotators of four social constructs—self‑esteem, self‑control, seeking belonging, and seeking recognition—in English song lyrics. It examines repeated‑measurement reliability, cross‑model convergence, and the transferability of consensus labels to supervised classification. Results show varying reliability across constructs, with self‑esteem being most stable and seeking recognition least stable, and indicate that consensus labels contain learnable signal for downstream tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Sep 3

Probing Cultural Signals in Large Language Models through Author Profiling

The study investigates cultural biases in large language models (LLMs) by testing their ability to perform author profiling—inferring singers’ gender and ethnicity—from song lyrics in a zero‑shot setting. Evaluating over 10,000 lyrics across several open‑source models, the authors find that most LLMs default toward North American ethnicity, while DeepSeek‑1.5B leans toward Asian ethnicity, and that Ministral‑8B exhibits the strongest ethnicity bias whereas Gemma‑12B is the most balanced. The paper introduces two fairness metrics, Modality Accuracy Divergence (MAD) and Recall Divergence (RD), to quantify these disparities and provides code and results publicly on GitHub and HuggingFace.

By Valentin Lafargue, Ariel Guerra-Adames, Emmanuelle Claeys, Elouan Vuichard, Jean-Michel Loubes

llms fine-tuning safety

arXiv AI

Jun 30

A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment

arXiv:2606. 29273v1 Announce Type: cross Abstract: Emotion recognition of song lyrics is a challenging task since lyrics may not necessarily align with the overall emotion of a song.

By Rashini Liyanarachchi, Frank Tran, Md Mahmudul Hasan, Aditya Joshi, Erik Meijering

llms safety

arXiv AI

Jul 31

VideoNorms: Benchmarking Cultural Awareness of Video Language Models

arXiv:2510. 08543v2 Announce Type: replace-cross Abstract: As Video Large Language Models (VideoLLMs) are deployed globally, it is important to assess their ability to reason across cultural contexts.

By Nikhil Reddy Varimalla, Yunfei Xu, Meng Fan Wang, Arkadiy Saakyan, Smaranda Muresan

llms benchmarks

arXiv Machine Learning

Oct 3

TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

TrueMuse is a new benchmark designed to evaluate data attribution in text-to-music models. It consists of a controlled dataset created by fine‑tuning three diffusion‑based models on curated attribution samples, providing known attribution targets. The benchmark covers four settings—melodic structure, timbral characteristics, artist‑level style, and genre‑level patterns—across 133 attributes, 648 models, and 95,456 generated samples, and is used to assess existing black‑box attribution methods along several dimensions.

By Jiawei Yu, Jian Liu

diffusion fine-tuning benchmarks

arXiv Computation and Language

Sep 1

Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features

arXiv:2608.28986v1 Announce Type: new Abstract: LLMs often struggle with modern Korean poetry, producing outputs that resemble "line-broken prose." We address two coupled tasks: detecting whether a K...

By Keunhyeung Park, Seunguk Yu, YoungBin Kim

llms

arXiv Computation and Language

Oct 19

Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs

arXiv:2610.22545v1 Announce Type: new Abstract: Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires...

By Maximilian Splieth\"over, Jannes L\"otfering, Dominik Bachmann, Alexander Espig, Willem Zuidema, Katrin Schulz, Henning Wachsmuth

llms fine-tuning