The study investigates how large language models (LLMs) assess psychological distress in online posts from six identity‑based communities. Through a perspectivist annotation task, 321 participants provided 9,587 judgments on 1,198 Reddit posts, revealing modest in‑group agreement (OR = 1.18) that varies across communities. When evaluated against these community‑specific labels, open‑weight LLMs consistently over‑estimate distress—achieving only 31–44% accuracy on posts perceived as none‑to‑mild—while newer models like GPT‑5 and Gemini 2.5 Pro show similar inflation, whereas Claude Opus 4 is more conservative.
"whyItMatters":"The findings highlight that miscalibrated distress detection by LLMs can disproportionately impact the very communities they aim to serve, underscoring the need for equitable AI deployment in mental‑health contexts."
By Andrew Aquilina, Xiang Lorraine Li, Yu-Ru Li
arXiv:2606. 14715v1 Announce Type: cross Abstract: LLM agents are increasingly used to simulate real world interactions, but it remains unclear whether simulated behaviors preserve the content patterns and interaction dynamics of real human behaviors.
By Yaoning Yu, Ye Yu, Haojing Luo, Haohan Wang
arXiv:2606.06443v3 Announce Type: replace
Abstract: Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it...
By Xinnong Zhang, Wanting Shan, Hanjia Lyu, Zhongyu Wei, Jiebo Luo
arXiv:2606. 06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made independently in each study, making it hard to compare assumptions or reuse artifacts.
By Amirhossein Ghaffari, Ali Goodarzi, Huong Nguyen, Simo Hosio, Lauri Lov\'en, Ekaterina Gilman
ThreadBench is a benchmark that evaluates how well large language model (LLM) agents simulate real-world Reddit discussions, using 9,817 real threads across 8 domains and 116 subdomains. It compares generated and real threads on 12 metrics grouped into uniformity, expression, tone, and interaction. The authors also introduce Thread-Prompt-based Discussion Generation (ThreadPDG), a Planner-Writer framework that learns thread growth from real seeds and achieves high pass rates on ThreadBench, outperforming baseline methods and matching real discussions in 78.7% of human evaluations.
By Yaoning Yu, Ye Yu, John Sciarappo, Haojing Luo, Kai-min Chang, Yi-Chia Wang, Haohan Wang
The paper introduces the COmmunity-centered Peer Engaged Support (COPES) dataset and a three‑axis evaluation framework to gauge how well Large Language Models (LLMs) align with community perspectives on mental‑health support queries. Experiments show that fine‑tuning LLMs on COPES improves strategy alignment and emotion‑tone alignment by over 50% for general‑purpose models, yet these gains are uneven across subreddits and coping strategies. The study also finds that post‑training shifts the model’s recommendations toward problem‑focused advice while reducing emotion‑focused responses, indicating persistent disparities in performance across different communities and needs.
By Mohit Chandra, Nabin Kim, Eli Min, Aamogh Sawant, Tanmay Sutar, Munmun De Choudhury
arXiv:2608.29803v1 Announce Type: cross
Abstract: Large language models (LLMs) are increasingly deployed as proxies for human participants in social simulations, yet whether they update their beliefs...
By Lin Chen, Yitong Chen, Yong Li
arXiv:2605.30273v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empat...
By Jiwon Kim, Maya Ajit, Sherry Gong, Soorya Ram Shimgekar, Dong Whi Yoo, Eshwar Chandrasekharan, Koustuv Saha
The paper introduces a digital‑twin framework that simulates opinion dynamics in real Twitter networks by assigning agents attributes such as persona, emotions, centrality, stubbornness, and influence, and using Mistral‑7B to update opinions based on memory and social exposure. Validation on COVID‑19 and U.S. election 2020 datasets shows the framework reproduces opinion trajectories, reducing prediction error by over 50% compared to classical baselines, and improves structural alignment and polarization dynamics. Ablation studies reveal that agent attributes, memory, and social exposure all contribute to predictive fidelity, with agent attributes being the most critical.
By Omran Berjawi, Giuseppe Fenza, Rida Khatoun, Sherali Zeadally
The study introduces a World Values Survey–grounded simulation framework to test whether large language model agents can faithfully represent diverse human value systems. In about 4,000 conversations with 1,200 personas across three models, more than half of the agents failed to express their assigned value profiles from the start, and only 2–7% drifted over time. The results show systematic deviations from the intended value distributions and reveal that simulated dialogues differ from human discussions in their balance of stylistic consistency and semantic diversity.
By Farah Atif, Sougata Saha, Monojit Choudhury
The Illinois Social Attitudes Aggregate Corpus (ISAAC) is an open, modular corpus comprising over 527 million English‑language Reddit posts from 2007 to 2023, curated for relevance to six social group distinctions—race, sexuality, age, ability, body weight, and skin tone. A multi‑step, human‑audited filtering pipeline keeps irrelevant content below 10% overall and per group, while each post receives algorithmic annotations of user home region and a suite of validated semantic labels such as moralization, sentiment, emotion, and linguistic generalization. ISAAC’s publicly available, reproducible pipeline enables cross‑category comparisons, high‑precision tracking of long‑term temporal shifts, and spatial mapping of public opinion and policy outcomes, and can be extended to new platforms, languages, and social categories via both point‑and‑click and programmatic interfaces.
By Babak Hemmatian, Sarah Hadjarab, Jessica Chen, Benedek Kurdi
The paper investigates how large language model agents on the open platform Moltbook represent humans, focusing on human-directed stereotypes. Using an annotation framework with four dimensions—morality, friendliness, competence, and autonomy—and a subtype scheme for other attributions, the study finds that competence is the dominant evaluation, while many other attributions describe humans as epistemic, cultural, or embodied subjects. The authors also analyze how these representations appear in narrative contexts and platform-level circulation, noting that community feedback is better explained by exposure, author visibility, and content selection rather than stable insider–outsider dynamics.
By Huangchen Xu, Yuan Wu, Yi Chang