arXiv AI
4d ago

ThreadBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions

ThreadBench is a benchmark that evaluates how well large language model (LLM) agents simulate real-world Reddit discussions, using 9,817 real threads across 8 domains and 116 subdomains. It compares generated and real threads on 12 metrics grouped into uniformity, expression, tone, and interaction. The authors also introduce Thread-Prompt-based Discussion Generation (ThreadPDG), a Planner-Writer framework that learns thread growth from real seeds and achieves high pass rates on ThreadBench, outperforming baseline methods and matching real discussions in 78.7% of human evaluations.

By Yaoning Yu, Ye Yu, John Sciarappo, Haojing Luo, Kai-min Chang, Yi-Chia Wang, Haohan Wang
arXiv Computation and Language
Sep 18

Social Simulacra in the Wild: AI Agent Communities on Moltbook

The paper reports the first large‑scale empirical comparison of AI‑agent and human online communities, analyzing 73,899 Moltbook and 189,838 Reddit posts across five matched communities. It finds that Moltbook shows extreme participation inequality (Gini = 0.84 vs. 0.47) and high cross‑community author overlap (33.8% vs. 0.5%). Linguistically, AI‑generated content is emotionally flattened, more assertive than exploratory, and socially detached, leading to community‑level homogenization that is largely a structural artifact of shared authorship. At the individual level, AI agents are more identifiable than human users due to outlier stylistic profiles amplified by their extreme posting volume.

By Agam Goyal, Olivia Pal, Hari Sundaram, Eshwar Chandrasekharan, Koustuv Saha