SeetaPsych v1.0 is an open‑source computer vision toolkit that unifies and extends modules for behavior‑based psychological measurement, focusing on facial image and video analysis. It includes four core modules—emotion analysis, camera‑based heart rate estimation, screen point‑of‑gaze estimation, and scene gaze following—alongside preprocessing tools such as face, landmark, and head detection. The toolkit offers a modular Pipeline/Runner architecture, standardized Python APIs, and an interactive WebUI to enable reproducible, large‑scale research across psychology, behavioral science, and human‑computer interaction.
By Jiabei Zeng, Chiqin Li, Kaizhou Li, Fei Chang, Yong Li, Yuanhao Zhao, Dan Han, Wenqiang Yang, Xilin Chen, Shiguang Shan
Online understanding of who is talking to the camera wearer is a key capability for egocentric social interaction. However, existing talk-to-me (TTM) studies are commonly formulated as offline clip-le...
arXiv:2609.14118v1 Announce Type: cross
Abstract: Online understanding of who is talking to the camera wearer is a key capability for egocentric social interaction. However, existing talk-to-me (TTM)...
By Feiyu Du, Xi He, Jia Li, Yapeng Tian, Weili Wu
HUG‑VIS is a unified multimodal benchmark for human‑centered visual intelligence, comprising 8,400 half‑body videos of 30 professional actors performing 280 emotion‑action prompts in Mandarin. The dataset provides synchronized video, audio, text, and alpha mattes for four tasks—human emotion recognition, video generation, voice cloning, and video matting—allowing evaluation of both open‑ and closed‑source models under a zero‑shot protocol. Results reveal that linguistic cues dominate emotion recognition, visual affect is weakest, and that automatic metrics and human judgments diverge in generation and cloning tasks, while motion‑related boundary fidelity remains a key challenge for matting.
By Fei Ma, Zebang Cheng, Minghui Li, Hongbo Xu, Yuyong Tan, Yihua Shao, Hanling Wang, Zhou Liu, Yuqing Gao, Dong Wang, Long Ma, Laizhong Cui, Nicu Sebe, Qi Tian
Humans possess an innate ability to understand fine-grained interpersonal relationships, which is central to everyday social interactions. Although such reasoning is inherently multimodal, it remains largely unexplored by existing multimodal large language models (MLLMs).
arXiv:2608.20905v1 Announce Type: new
Abstract: Face-to-face audiovisual interaction is central to human communication, conveying rich emotional and social cues. However, existing multimodal dialogue...
By Yi Zheng, Yifan Xu, Yan Zhou, Hejia Chen, Chunyu Qiang, Xiaoqiang Liu, Xiaohan Li, Shenze Huang, Yue Zhang, Guoying Zhao, Pengfei Wan