arXiv AI

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores

arXiv:2605. 12530v2 Announce Type: replace-cross Abstract: LLM fairness should be evaluated through in-situ behavioral pattern rather than standardized-test Q&A benchmarks.

arXiv AI

Jun 16

AgentFairBench: Do LLM Agents Discriminate When They Act?

arXiv:2606. 16723v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is still measured by grading answers.

By Triveni Morla, Rohith Reddy Bellibaltu, Manpreet Singh, Manmeet Singh Kapoor

llms agents benchmarks safety

arXiv AI

Jul 2

Large language models replicate and predict human cooperation across experiments in game theory

arXiv:2511. 04500v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in high-stakes domains and as imitators of human behavior in the social and behavioral sciences.

By Andrea Cera Palatsi, Samuel Martin-Gutierrez, Ana S. Cardenal, Max Pellert

llms agents safety

arXiv AI

Jun 18

Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts

arXiv:2606. 18258v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining boundaries.

By Sunnie S. Y. Kim, Margit Bowler, Leon A Gatys

llms

arXiv Machine Learning

Aug 7

Conditional Cognitive Biases in LLMs: How Biased User Turns Modulate In-Context Reasoning

arXiv:2608. 05166v1 Announce Type: cross Abstract: We present an evaluation of cognitive bias expression in state-of-the-art instruction-tuned LLMs under realistic multi-turn interaction settings.

By Sachini Weerasekara, Sagar Kamarthi, Jacqueline Isaacs

llms benchmarks safety

arXiv AI

Jun 30

LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals

arXiv:2411. 10109v3 Announce Type: replace Abstract: Machine learning can predict human behavior well when substantial structured data are available for well-defined outcomes.

By Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein

llms agents benchmarks

arXiv AI

Jun 12

Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior

arXiv:2606. 12730v1 Announce Type: new Abstract: Anticipating LLM behavioral tendencies from low-cost psychometric probes is critical for safe deployment, but only if self-reports (SR) reliably predict behavior.

By Rafal Kocielnik, Pengrui Han, Peiyang Song, Myrl G. Marmarelis, Ramit Debnath, Dean Mobbs, Anima Anandkumar, R. Michael Alvarez

llms safety

arXiv AI

Jul 7

Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings

arXiv:2601. 11049v2 Announce Type: replace-cross Abstract: We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load.

By Stephen Pilli, Vivek Nallur

llms agents safety

arXiv AI

Jun 30

Collective cooperation without individual fidelity in LLM agents

arXiv:2606. 30454v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making.

By Henrique Ferraz de Arruda, Carlos Gracia L\'azaro, Alberto Aleta, Yamir Moreno

llms agents benchmarks

arXiv Machine Learning

Aug 10

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509. 16462v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economic disparities.

By Mina Arzaghi, Alireza Dehghanpour Farashah, Florian Carichon, Jean-Fran\c{c}ois Plante, Golnoosh Farnadi

llms rag fine-tuning benchmarks safety

arXiv AI

Jul 29

LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation

arXiv:2607. 24780v1 Announce Type: new Abstract: Evaluating frontier LLMs is challenging: static benchmarks suffer from contamination and saturation -- leaving users unable to distinguish top models and developers blind to specific failure modes -- while human preference is subjective.

By Xingyu Chen, Rui Wang, Zhaopeng Tu, Liefeng Bo

llms reinforcement-learning benchmarks

arXiv AI

Aug 12

Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation

arXiv:2608. 10385v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as relevance assessors in information retrieval (IR) evaluation, raising questions about how assessor framing affects judgment reliability and downstream system comparison.

By Samaneh Mohtadi, Pietro Bernardelle, Joel Mackenzie, Gianluca Demartini

llms rag

arXiv Machine Learning

Jun 5

Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability

arXiv:2605. 03217v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased.

By Yash Aggarwal, Atmika Gorti, Vinija Jain, Aman Chadha, Krishnaprasad Thirunarayan, Manas Gaur

llms efficiency safety

Related stories