arXiv AI August 18, 2026 By Mantas Lukauskas, Viktorija \v{S}arkauskait\.e

Plausible but Not Valid: A Psychometric Audit of LLMs as Synthetic Survey Respondents

arXiv:2608. 14606v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as synthetic survey respondents, but existing evaluations ask whether answers look plausible at the individual level.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 22

Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents

arXiv:2609.22090v1 Announce Type: new Abstract: An LLM producing the response pattern associated with a human psychological effect is not the same claim as the LLM possessing that bias. We present Ps...

By Joy Bose

llms agents robotics benchmarks safety

arXiv AI

Jun 19

Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact

arXiv:2606. 20205v1 Announce Type: new Abstract: Psychological instruments designed for humans are increasingly used to assign large language models (LLMs) stable psychological profiles that affect their usability, safety assessment, and use as proxies for human participants in research.

By Jelena Meyer, David Garcia, Dirk U. Wulff

llms safety

arXiv AI

Jul 8

An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models

arXiv:2606. 09843v3 Announce Type: replace-cross Abstract: Large language models (LLMs) give stable answers to personality questionnaires, yet these self-reports fail to predict how the models behave.

By Juan Manuel Contreras

llms safety

arXiv AI

Jun 10

An LLM-Native Psychometric Instrument Does Not Predict LLM Behavior: Evidence Across 25 Models

arXiv:2606. 09843v1 Announce Type: cross Abstract: Large language models (LLMs) produce stable self-reports on personality inventories, but these self-reports do not predict observed behavior.

By Juan Manuel Contreras

llms safety

arXiv Computation and Language

Sep 1

LLM Judges as Raters: A Pre-Registered Audit of Severity, Halo, Reliability, and Version Instability in LLM Essay Scoring on Public Corpora

arXiv:2608.29517v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as essay graders in learning analytics, evaluated almost exclusively with agreement statistics. Educ...

By Veerendra Kumar Sunkavalli

llms benchmarks

arXiv Computation and Language

Sep 10

ActTraitBench: Quantifying the Knowledge-Decision Gap in Large Language Models via Human-Grounded Behavioral Validation

arXiv:2605.29791v2 Announce Type: replace Abstract: While Large Language Models (LLMs) can convincingly simulate personas in explicit self-reports, they often deviate in implicit behavioral decisions...

By Yutong Yang, Chenxi Miao, Weikang Li, Yunfang Wu

llms benchmarks safety

One email a morning, machine-written

Related stories

Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents

Apparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact

An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models

An LLM-Native Psychometric Instrument Does Not Predict LLM Behavior: Evidence Across 25 Models

LLM Judges as Raters: A Pre-Registered Audit of Severity, Halo, Reliability, and Version Instability in LLM Essay Scoring on Public Corpora

ActTraitBench: Quantifying the Knowledge-Decision Gap in Large Language Models via Human-Grounded Behavioral Validation