ThreadBench is a benchmark that evaluates how well large language model (LLM) agents simulate real-world Reddit discussions, using 9,817 real threads across 8 domains and 116 subdomains. It compares generated and real threads on 12 metrics grouped into uniformity, expression, tone, and interaction. The authors also introduce Thread-Prompt-based Discussion Generation (ThreadPDG), a Planner-Writer framework that learns thread growth from real seeds and achieves high pass rates on ThreadBench, outperforming baseline methods and matching real discussions in 78.7% of human evaluations.
arXiv:2606. 14715v1 Announce Type: cross Abstract: LLM agents are increasingly used to simulate real world interactions, but it remains unclear whether simulated behaviors preserve the content patterns and interaction dynamics of real human behaviors.
arXiv:2608. 09790v1 Announce Type: new Abstract: Online credit card discussions provide a natural setting for studying how consumers communicate about financial products.
arXiv:2610.20360v1 Announce Type: new Abstract: Gossip, defined as evaluative talk about absent third parties, is widely seen as a uniquely human social practice. This study examines whether comparab...
arXiv:2606. 06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made independently in each study, making it hard to compare assumptions or reuse artifacts.
arXiv:2610.14570v1 Announce Type: new Abstract: Large language models (LLMs) support social simulation, but faithfully reproducing real users requires grounding both structured platform decisions and...
arXiv:2601. 19921v2 Announce Type: replace-cross Abstract: Multi-agent debate (MAD) is widely used to improve large language model (LLM) performance through test-time scaling, yet recent work shows that vanilla MAD often underperforms simple majority vote despite higher computational cost.
arXiv:2605.27388v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used as proxies for computational social analysis, yet faithfully representing the "thick description...
arXiv:2601. 02854v2 Announce Type: replace Abstract: As an agent-level reasoning and coordination paradigm, Multi-Agent Debate (MAD) orchestrates multiple agents through structured debate to improve answer quality and support complex reasoning.
arXiv:2605.12894v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are uncle...
DialFork is a new benchmark that evaluates procedural hallucination in conversational agents, a failure mode where agents produce fluent, factually plausible utterances but deviate from the intended dialogue process. The benchmark consists of 36,278 dialogues across 24 tasks in 12 domains, created by forking conformant dialogues and injecting one of four hallucination types, with validation by 30 human annotators. Three evaluation tasks are defined—monitoring overall deviation, procedural retrieval of a single dialogue, and real‑time detection of off‑track turns—using dialog‑flow‑aware metrics FlowKL and FlowScore, which outperform baseline semantic similarity metrics.
The paper introduces CoCoEval, a framework for evaluating large language model (LLM)–simulated conversations by detecting 10 types of inconsistent and uncollaborative behaviors at the turn level. Using CoCoEval, the authors compare human conversations with those generated by GPT‑4.1, GPT‑5.1, and Claude Opus 4, finding that LLMs produce far fewer such behaviors under vanilla prompting and that prompt engineering or fine‑tuning often over‑produces specific behaviors. The study highlights gaps between human and LLM‑simulated interactions that conventional Likert‑scale evaluations miss, raising concerns about using LLMs as proxies for human social interaction.
The paper introduces R$^2$-MAD, a framework that enhances multi-agent debate by giving agents an experience memory from past debates. It uses a debate-state-aware retrieval policy to adjust concept priors based on current consensus, and derives confidence weights from retrieved experiences to modulate peer influence. Experiments demonstrate consistent improvements over existing single-agent and MAD baselines.