arXiv Computer Vision

STC-MGNet: Spatial-Temporal-Channel Attention-Enhanced Multimodal Graph Network for Pedestrian Intention Prediction

The paper introduces STC-MGNet, a spatial‑temporal‑channel attention‑enhanced multimodal graph network designed to predict pedestrian crossing intentions. It employs a pose graph encoder with three attention branches that focus on skeletal joints, observation frames, and feature channels, and integrates visual and ego‑vehicle motion data through a multi‑stage cross‑modal channel modulation mechanism. Experiments on JAADbeh, JAADall, and PIE datasets show accuracy rates of 70%, 87%, and 88%, surpassing several baseline methods.

arXiv Computer Vision
Sep 11

TrajFusionNet+: Transformer-Based Prediction of Pedestrian Crossing Intention via Fusion of Trajectory Representations and Scene Graphs

TrajFusionNet+ is a transformer-based model that predicts pedestrian crossing intention by fusing sequential trajectory data, visual trajectory overlays, and graph-based scene context. It extends the earlier TrajFusionNet with three attention modules—Sequence, Visual, and Graph—to capture temporal, visual, and relational cues. The model outperforms state‑of‑the‑art methods on the PIE and JAAD datasets and shows better generalization under a joint‑training, separate‑evaluation protocol.

By Fran\c{c}ois G. Landry, Moulay A. Akhloufi
arXiv Computer Vision
Sep 18

MTF-Net: Multi-Modal Temporal Feature Fusion Network for Pedestrian Intention Prediction

MTF‑Net is a Multi‑Modal Temporal Feature Fusion Network that jointly models kinematic, appearance, and contextual cues for pedestrian intention prediction. It fuses four modalities—bounding‑box dynamics, human pose keypoints, local context, and scene‑level semantics—within a recurrent framework enhanced by gated linear units (GLUs) and an attention‑guided fusion head. Evaluations on the PIE and JAAD benchmarks show that MTF‑Net outperforms recent transformer‑ and graph‑based models, achieving up to 0.95 AUC on PIE and 0.94 AUC on JAAD while maintaining real‑time performance.

By Md Mahfuzur Rahman, Pengzhan Zhou, A. F. M. Abdun Noor, Md Imam Ahasan, Md Mustafizur Rahman, Fang Qu
arXiv AI
Aug 25

Multi-Context Fusion Transformer for Pedestrian Crossing Intention Prediction in Urban Environments

The paper introduces the Multi-Context Fusion Transformer (MFT), a model that predicts pedestrian crossing intentions in urban settings by integrating four types of contextual information—pedestrian behavior, environment, localization, and vehicle motion—through a progressive fusion strategy. MFT uses intra-context attention for reciprocal interactions within each context, cross-context attention to combine these contexts into a global representation, and guided attention mechanisms to refine both context tokens and the global token. Experiments on JAADbeh, JAADall, and PIE datasets show MFT outperforms existing methods with accuracies of 73%, 93%, and 90% respectively, and ablation studies confirm the importance of each network component and input context.

By Yuanzhe Li, Hang Zhong, Steffen M\"uller
arXiv AI
Aug 13

HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

arXiv:2608. 12187v1 Announce Type: cross Abstract: Transformer-based methods have achieved strong performance in monocular 3D human pose estimation, but most existing approaches organise spatial and temporal reasoning as separate stages, which may weaken unified spatial-temporal interdependencies inherent in human motion and compress frame-level structural information before temporal modelling.

By Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin, Amir Atapour-Abarghouei
arXiv Machine Learning
Jul 24

HERMES: Heterogeneous Edge-Relational Multi-Head Embedded SSM Attention for Traffic Conflict Prediction at Signalized Intersections

arXiv:2607. 20505v1 Announce Type: cross Abstract: Surrogate safety measures (SSMs) enable proactive traffic safety assessment, but many existing methods evaluate pairwise interactions independently or flatten multi-agent scenes into fixed feature vectors, limiting their ability to represent heterogeneous interaction structure and evolving scene-level risk.

By Md Monzurul Islam, Subasish Das
arXiv AI
Sep 15

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation

The paper presents an end‑to‑end system that converts driving footage into dynamic vision sensor (DVS) event streams, augments training with simulated DVS data, and trains a convolutional spiking neural network (Conv‑SNN) to classify pedestrian crossing intent as crossing or non‑crossing. The Conv‑SNN, trained with a class‑balanced loss and surrogate‑gradient learning, achieves high accuracy and F1 scores on JAAD and CARLA DVS datasets, outperforming or matching prior frame‑based methods while operating on sparse temporal representations. The study details architectural choices, neuron dynamics, and training protocols, and provides a convergence analysis and domain‑transfer evaluation.

By Henok Teklu, Mustafa Sakhai, Maciej Wielgosz, Matej Mertik
arXiv AI
5d ago

SOMA: A Benchmark for Fine-Grained Egocentric Behavior and Body-Motion Understanding

SOMA is a new benchmark that tests fine‑grained egocentric body‑motion understanding, featuring 1,632 human‑verified questions across body configuration, motion dynamics, and temporal reasoning. The study evaluates 37 multimodal large language models (MLLMs) and finds that humans achieve 68.5% accuracy while the best MLLM (Qwen3.6‑27B) reaches only 50.0% zero‑shot accuracy. The authors also provide a training dataset and a multimodal baseline to help close the performance gap and highlight common failure modes such as misreading body parts and temporal errors.

By Matea Tashkovska, Na\"el Ouerghemi, Haozhe Qi, Alexander Mathis