arXiv Computer Vision

Rate as a Feature, Not a Nuisance: Curriculum FiLM Training for Event-Based Graph Neural Networks

arXiv AI
5d ago

Smart Surveillance: Deep Learning and Hybrid Approaches for Dynamic Scene Analysis, Object Detection and Motion Tracking

The paper evaluates event-triggered recording strategies for static surveillance cameras, comparing six motion-based triggers, a CNN detector, strided detection, and hybrid approaches on a benchmark of seven scenes. Detector-based triggers reduce idle frame storage to 19‑28% versus 36‑89% for motion triggers, and remain robust under camera jitter, keeping under 29% idle frames. In addition, the study shows that increasing YOLO11n input resolution improves detection accuracy more than larger models, and that longer track memory boosts tracking IDF1, though detection remains the limiting factor.

By Shahran Rahman Alve
arXiv Computer Vision
Sep 28

The Shape of Events: Edge-Based Inductive Biases via Cross-Domain Distillation

The paper investigates how knowledge distillation from event cameras to RGB images can alter the inductive biases of convolutional neural networks. By transferring learning from the event domain, the authors find that models gain color invariance, a shape bias, and improved robustness to high‑frequency noise, largely due to reduced reliance on texture and increased emphasis on edge‑based object shape. These changes are evidenced by early‑layer processing differences and a spectral trade‑off between robustness to missing high‑frequency content and vulnerability to its contamination or geometric disruption.

By Soshun Kihara, Shunsuke Yasuki, Masato Taki
arXiv Computer Vision
Sep 15

A 25-$\mu$s/inf Event-driven Graph Neural Network Processor with Spatiotemporal Caching and Spline Convolution for Ultra-low-latency AI at the Edge

The paper introduces ETHEREAL, the first accelerator for event-driven graph neural networks (EV‑GNNs) that can handle 640×480 resolution inputs. It achieves this through a neighbor‑parallel spline convolution engine and a 2D/3D‑split memory hierarchy that includes a novel region‑of‑interest spatiotemporal caching mechanism. Measurements show end‑to‑end inference latency of 25.6 µs and energy consumption of 1.7 µJ per event on state‑of‑the‑art workloads.

By Adrian Kneip, Martin Lefebvre, Daniel Gehrig, Victoria Catal\'an Pastor, Davide Scaramuzza, Marian Verhelst, Charlotte Frenkel
arXiv Computer Vision
5d ago

Readout and Operating-Point Sensitivity in Event-Stream Candidate Selection

The paper evaluates how readout settings and operating points affect event-stream candidate selection in event-triggered recording. Using 520 DailyDVS‑200 clips and 47 participants, the authors compare several encoder configurations—including DINO‑CLS, patch mean, and MobileNetV3‑Small—across different accumulation durations and view counts, reporting metrics such as target capture, false triggers, and partial AUC. The study finds that lightweight baselines and specific operating points significantly influence performance, while no clear advantage is observed for the JEPA‑specific method.

By Yifan Shuai
arXiv AI
Sep 15

Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation

The paper presents an end‑to‑end system that converts driving footage into dynamic vision sensor (DVS) event streams, augments training with simulated DVS data, and trains a convolutional spiking neural network (Conv‑SNN) to classify pedestrian crossing intent as crossing or non‑crossing. The Conv‑SNN, trained with a class‑balanced loss and surrogate‑gradient learning, achieves high accuracy and F1 scores on JAAD and CARLA DVS datasets, outperforming or matching prior frame‑based methods while operating on sparse temporal representations. The study details architectural choices, neuron dynamics, and training protocols, and provides a convergence analysis and domain‑transfer evaluation.

By Henok Teklu, Mustafa Sakhai, Maciej Wielgosz, Matej Mertik
arXiv Computer Vision
Sep 30

TT-VidT: Decoupling the Temporal Axis for Efficient Motion-Centric Video Pretraining

TT-VidT is a video pretraining method that decouples the temporal axis by combining a per‑frame ViT-B/16 spatial encoder with a compact Temporal Transfer Layer trained via Diff Compression. The authors conduct a systematic 24‑configuration study to isolate architecture, objective, and decoder effects, showing that the full TT-VidT design yields the strongest motion‑sensitive representations. In downstream fine‑tuning, TT‑VidT outperforms state‑of‑the‑art baselines on Jester, Something‑Something V2, ARID, and Diving48 while using significantly fewer encoder FLOPs.

By Shih-Ying Yeh, Daniel Z. Kaplan, Xuehai Wang, Fu-En Yang, Min-Hung Chen, Shang-Hong Lai
arXiv AI
Aug 28

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.

By Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner