The paper introduces the Mult2EMo dataset, which gathers annotations from both authors and readers on multimodal social media posts and the real‑world events that triggered them. It investigates how well readers can reconstruct the authors’ emotional experience from the post content, emphasizing the importance of both text and image modalities. The study finds that accurate emotion reconstruction is possible but remains challenging, especially when images dominate the expression and when understanding the triggering event is essential.
By Christopher Bagdon, Carina Silberer, Roman Klinger
The paper reviews how large language models are applied in mental health, covering areas such as social media analysis, clinical conversational agents, therapy support tools, prompt engineering, and multimodal learning. It synthesizes interdisciplinary studies that use social media posts, electronic medical records, and multimodal inputs to detect depression, assess suicide risk, provide personalized therapy, and generate psychoeducational content. The review also discusses advances in model interpretability, annotation strategies, multimodal fusion techniques, and highlights ethical, sociotechnical, and regulatory challenges while proposing frameworks for safe, equitable, and accountable deployment.
By Yisong Chen, Yifan Gao, Sijing Yu, Chuqing Zhao, Yang Lu
arXiv:2610.33043v1 Announce Type: cross
Abstract: Short-form video combines speech, imagery, and on-screen text, creating challenges for systematic content analysis. We present a five-stage multimoda...
By Julie R. Ancis, Michael Fire, Omer Tarshish
The paper presents a scalable approach to harmful content moderation on social media by leveraging large language models (LLMs) for few-shot, in-context learning. Experiments across multiple LLMs show that this method outperforms proprietary baselines such as Perspective and OpenAI Moderation, as well as prior few-shot learning techniques, in detecting harmful content. The study also explores the addition of visual cues like video thumbnails to assess multimodal improvements, highlighting the advantages of LLM-based moderation for dynamic and large-scale content filtering.
By Akash Bonagiri, Lucen Li, Rajvardhan Oak, Zeerak Babar, Magdalena Wojcieszak, Anshuman Chhabra
The paper presents a new taxonomy for online harm on video platforms, expanding existing frameworks to multimodal contexts and defining six content-based categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. It also evaluates multimodal large language models (MLLMs) as alternative annotators, comparing GPT‑4‑Turbo, non‑trained crowdworkers, and trained human annotators on a dataset of 19,422 potentially harmful YouTube videos. The study finds that MLLMs can perform comparably to human annotators in both binary and multi‑label classification tasks, suggesting potential applications in content moderation and social science research.
By Claire Wonjeong Jo, Miki Weso{\l}owska, Magdalena Wojcieszak
arXiv:2601. 05232v3 Announce Type: replace-cross Abstract: Most people now get their news from videos on social media, such as YouTube and Facebook, rather than through curated journalism.
By P. Gilda (Columbia University), P. Dungarwal (Columbia University), A. Thongkham (Columbia University), E. T. Ajayi (St John's University), S. Choudhary (Columbia University), T. M. Terol (Columbia University), C. Lam (Columbia University), J. P. Araujo (Columbia University), M. McFadyen-Mungalln (Columbia University), L. S. Liebovitch (Columbia University), P. T. Coleman (Columbia University), H. West (Columbia University), K. Sieck (Toyota Research Institute), S. Carter (Toyota Research Institute)
arXiv:2609.24215v1 Announce Type: new
Abstract: Although text-to-image models can accurately depict subjects and scenes, creators still struggle to specify the fine-grained emotions an image should c...
By Minglang Li, Yueyue Fang, Xieping Gao
The Illinois Social Attitudes Aggregate Corpus (ISAAC) is an open, modular corpus comprising over 527 million English‑language Reddit posts from 2007 to 2023, curated for relevance to six social group distinctions—race, sexuality, age, ability, body weight, and skin tone. A multi‑step, human‑audited filtering pipeline keeps irrelevant content below 10% overall and per group, while each post receives algorithmic annotations of user home region and a suite of validated semantic labels such as moralization, sentiment, emotion, and linguistic generalization. ISAAC’s publicly available, reproducible pipeline enables cross‑category comparisons, high‑precision tracking of long‑term temporal shifts, and spatial mapping of public opinion and policy outcomes, and can be extended to new platforms, languages, and social categories via both point‑and‑click and programmatic interfaces.
By Babak Hemmatian, Sarah Hadjarab, Jessica Chen, Benedek Kurdi
The study examines how emotions are represented across layers of large language models (LLMs) by probing eight 1B–9B open‑weight models on three datasets (Twitter, Reddit, autobiographical narratives). It finds that the optimal probing layer varies systematically with the dataset, moving from near‑input layers to deeper layers, and that targeted forward‑pass interventions on these layers degrade performance more than random interventions. Additionally, the selected layers transfer across datasets and emotion categories, and early‑exit representations from these layers outperform full‑depth exits by an average of 6.9 percentage points.
By Tian Fang, Ga\"el Guibon, Davide Buscaldi
arXiv:2606. 28334v1 Announce Type: cross Abstract: Recent advances in artificial intelligence (AI) and social media data have led to growing optimism about the ability to detect suicide risk at scale.
By Yaakov Ophir, Ofri Hefetz, Refael Tikochinski, Kfir Bar, Shir Lissak, Shulamit Grinapol, Haya Wachtel, Eyal Fruchter, Roi Reichart
Early stress detection and risk prediction are crucial for timely mental health assessment and support. While large language models have recently shown promise for social media stress prediction, rely...
The paper investigates whether out‑of‑the‑box pretrained Large Language Models (LLMs) can detect human affect by predicting continuous engagement annotations from video game footage. Using 80 minutes of annotated first‑person shooter gameplay from the GameVibe corpus, the authors run over 4,800 experiments to assess how LLM architecture, size, input modality, prompting strategy, and ground‑truth processing affect performance. Results show that while LLMs can outperform traditional machine‑learning baselines, they generally lag behind human‑provided continuous experience annotations, and the study explores reasons for performance variability and outlines directions for future automated emotion labeling.
By David Melhart, Matthew Barthet, Georgios N. Yannakakis