RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs
arXiv:2607. 05679v1 Announce Type: cross Abstract: Language models (LMs) exhibit problematic biases, such as stereotypes.
arXiv:2606. 03029v1 Announce Type: cross Abstract: A core goal of computational social science is to discover interpretable differences in how language varies across outcomes of interest, such as political affiliation or instructional quality.
arXiv:2607. 05679v1 Announce Type: cross Abstract: Language models (LMs) exhibit problematic biases, such as stereotypes.
The paper examines how large language models (LLMs) respond to different demographic cues—such as names—when users seek advice, focusing on race and gender in a U.S. context. It finds that using different cues for the same group leads to only partially overlapping changes in model responses, producing inconsistent conclusions about personalization and unstable bias metrics. The authors argue that LLMs react to linguistic signals tied to cues rather than to stable demographic categories, and they call for evaluations that use multiple cues and consider underlying mechanisms.
arXiv:2607. 19243v1 Announce Type: cross Abstract: Although Large Language Models (LLMs) demonstrate remarkable multilingual fluency, their internal knowledge representations remain disproportionately biased toward high-resource languages.
arXiv:2601.12868v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly operate in high-stakes settings where demographic attributes such as race and ethnicity may be expl...
arXiv:2605. 07409v2 Announce Type: replace-cross Abstract: Natural Language Processing is rapidly evolving into a primary instrument for Computational Social Science, with researchers increasingly using embeddings to measure latent constructs such as novelty, creativity, and bias.
arXiv:2510. 12857v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide.
arXiv:2608. 05726v1 Announce Type: cross Abstract: Large Language Models (LLMs) are often used as evaluators of text quality, known as LLM-as-a-Judge, which can outperform conventional automatic evaluation metrics that rely on reference texts.
The paper proposes a method to steer large language models (LLMs) to generate explanations tailored to specific target groups. It first identifies group-specific attributes related to explanatory style and knowledge, then uses activation engineering to compute steering vectors that are added to the LLM’s internal activations during inference. Experiments show that this attribute-based steering improves specificity and factuality of explanations compared to prompting and existing steering baselines, and a human study confirms better tailoring to target groups.
arXiv:2608.10503v2 Announce Type: replace Abstract: As Large Language Models (LLMs) are increasingly deployed as autonomous agents, accurately evaluating their latent values and biases is critical. T...
The paper introduces Debiased Inference with Multiple Imperfect Measurements (DMM), a framework that uses several error‑prone AI measurements to perform valid downstream statistical inference without requiring costly gold‑standard labels. By assuming conditional independence of the measurements given the true label and unit‑level features, DMM leverages CP decomposition and semiparametric theory to prove consistency and asymptotic normality of its estimator. Simulations demonstrate that DMM yields valid inference and can improve efficiency when additional imperfect measurements are available, and the authors provide diagnostics for the key independence assumption.
The paper maps the nascent social‑science literature on large language models (LLMs) by analysing 198 curated papers and 47,719 field‑scale papers. It identifies three main domains—LLM as Social Minds, LLM Societies, and LLM‑Human Interactions—each containing 13 subcategories such as reasoning, bias, collective intelligence, and trust. The taxonomy is validated through clustering stability, author classification agreement, and topic mapping, revealing differing prominence across conference and journal venues.
arXiv:2607. 24435v1 Announce Type: cross Abstract: Large language models may easily assign personality labels from text, but model interpretability remains an open problem.