arXiv Computation and Language

Detecting AI Impostors: How Do Middle Schoolers Identify LLM Agents in a Live Collaborative Setting?

arXiv AI
Sep 24

How Children Design and Reason about Trustworthy AI Chatbots

The study explores how children design AI chatbots and what they consider trustworthy. Using a custom chatbot-building environment, 115 learners aged 8‑18 created 119 chatbots and adjusted traits such as confidence, transparency, and formality. Findings show younger children equate trust with purpose‑fulfillment, while older children focus on transparent, calibrated design, and that students calibrate academic chatbots to be more formal and transparent than hobby ones.

By Deniz Ozturk, Jiayu Li, Daksh Pratap Singh, Yasitha Rajapaksha, Fasika Melese, Bahare Riahi, Shiyan Jiang, Qiao Jin, Joey Huang, Veronica Catet\'e, Tiffany Barnes, Xiaoyi Tian
arXiv AI
Sep 21

From Memory to Behavior: A Behavior-Aware Role-Playing Framework for Social Media Influencers

The paper introduces a Situation–Internal state–Behavior Persona method to improve large language models’ ability to impersonate real individuals in social media contexts. It also proposes an evaluation protocol that supplies LLM evaluators with reference information about the target individual. Experiments on a new dataset of social media replies show the method surpasses state‑of‑the‑art in‑context learning baselines, and the protocol correlates moderately with human judgments, while additional tests on fictional characters confirm broader applicability.

By Ji-Lun Peng, Yi-Zhen Zhang, Chun-Nan Chou, Yun-Nung Chen
arXiv Computation and Language
Sep 16

Towards Detecting AI-Assisted Responses in Online Surveys

The paper introduces ASURRE, a benchmark dataset for detecting AI‑assisted responses in online surveys. It evaluates how different LLM usage strategies—ranging from full generation to persona‑grounded agentic completion—affect the performance of existing machine‑generated text detectors. The study finds that while naive AI usage is easily detected, more sophisticated persona‑grounded agents approach chance performance, yet still leave identifiable behavioural traces that can be aggregated to improve detection.

By Qizhou Wang, Bogdan Mamaev, Christopher Leckie
arXiv AI
Sep 1

QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents

arXiv:2605.27068v2 Announce Type: replace-cross Abstract: Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Mo...

By Ye Yuan, Rui Song, Weien Li, Zeyu Li, Haochen Liu, Xiangyu Kong, Changjiang Han, Yonghan Yang, Zichen Zhao, Zixuan Dong, Fuyuan Lyu, Bowei He, Haolun Wu, Jikun Kang, Xue Liu