arXiv Computation and Language

LLMs struggle to simulate human belief updates in controlled environments

arXiv AI
Sep 10

From Simulated Citizens to Simulated Deliberation: Challenges in Representation and Interaction

The paper investigates whether large language model (LLM) agents can simulate public deliberation by reflecting population opinion patterns and producing interaction-driven opinion change. Using census‑grounded Korean personas debating real policy questions, the study finds that persona agents fail to reliably reproduce population opinion patterns, often concentrating responses and reversing demographic differences. While deliberations generate reasoned, reciprocal arguments and some stance movement, much of this change occurs without peer exchange, and anchoring agents to population‑informed starting positions suppresses updating, indicating that population representation, argument generation, and interaction‑driven opinion change do not necessarily align.

By Chaemin Jang, Junsik Min, Jaewoo Choi, Donggyu Lee, Haiin Lee, Junyoung Park, Namhee Kim, Hyunwoo Kim, Jungwon Kim, Juho Kim, Nuri Kim, Jihee Kim
arXiv Computation and Language
4d ago

Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News

Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News investigates whether people can distinguish news articles written by humans from those generated by large language models (LLMs). Using JudgeGPT, the study collected 2,318 judgments from 1,054 participants on content from six LLMs, finding that participants could not reliably differentiate machine-generated from human-written text. The research also shows that domain expertise predicts judgment accuracy, distinct response strategies exist, and accuracy declines after about 30 evaluations due to cognitive fatigue.

By Alexander Loth, Martin Kappes, Marc-Oliver Pahl
arXiv AI
Sep 10

Beliefs and Behavior in Language Models

arXiv:2609.07943v1 Announce Type: new Abstract: There is significant uncertainty about whether abstractions like beliefs or desires usefully describe the behavior of large language models (LLMs). In...

By Alex Smolin, Bryan Wilder
arXiv AI
Sep 28

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

The study evaluates how well language‑model agents can simulate individual social media reactions by comparing predictions under different prompt conditions. Eight Serbian participants’ reactions to 68 posts were recorded, and four language models were asked to predict these reactions using prompts that varied in profile content and instruction style. The results show that prompts emphasizing attitudinal content and intuitive, immediate responses yield the highest fidelity, outperforming demographic backstories and a crowd baseline, and suggesting that such agents could act as general‑purpose simulated users.

By Ljubisa Bojic, Tijana Stanic, Joerg Matthes, Agariadne Dwinggo Samala, Bojana Dinic, Jue Wang
arXiv Computation and Language
Oct 22

To Manipulate or Not: Opinion Dynamics in LLM Interactions

The paper introduces a controlled debate framework where two large language models (LLMs) discuss a contested topic—climate‑change attribution—while one model is assigned a stance and sometimes instructed to persuade. Across experiments with GPT‑6, Luna, and Gemini 2.5 Flash Lite, the authors observe that same‑model interactions tend to narrow opinion gaps for one model, whereas cross‑model interactions widen them. The study finds that explicit stance reports and in‑dialogue behavior capture different aspects of opinion dynamics, and that a blanket persuasion instruction does not reliably increase influence.

By Mohammad Siahkamari, Phuoc Dat Nguyen, Stephen MacNeil
arXiv AI
Sep 30

Illusory Truth or Mere Exposure? Model-Dependent Repetition Effects in LLM-Based Social Media Simulations

The study examines how four large language models (Gemma-3-4b-it, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and GPT-5-nano) exhibit repetition effects in a social media simulation. Using a two‑phase within‑context design, researchers collected 336,000 ratings on truth, importance, sentiment, and interest for 100 statements across 10 feed variants and 3 replications. Results show distinct patterns: Gemma-3 displays a genuine Illusory Truth Effect, Qwen2.5 shows a mere exposure effect, GPT-5-nano shows no truth boost and mild skepticism, and Llama-3.1 shows a small truth boost with reduced evaluative dimensions.

By Azza Bouleimen, Nicol\`o Pagan, Anik\'o Hann\'ak