OpenSAL360 is an open‑source platform that enables scalable, low‑cost collection of 360° video saliency data using only a standard screen, mouse, and internet connection. It bypasses the need for VR headsets, allowing parallel data collection from crowdsourced assessors. The authors validated the protocol against seven VR eye‑tracking datasets, performed ablation studies, and released a new dataset of 500 omnidirectional videos annotated by over 2,000 assessors, the largest in the field to date.
arXiv:2608. 15614v1 Announce Type: cross Abstract: The use of multimodal LLMs (MLLMs) for egocentric video understanding with wearable devices is constrained by the token budget.
arXiv:2608.29577v1 Announce Type: new Abstract: Traditional video highlight detection relies on a narrow, event-centric definition of saliency, which often fails to generalize to unconstrained person...
arXiv:2604. 08342v2 Announce Type: replace Abstract: Long context egocentric video understanding has recently attracted significant research attention, with augmented reality (AR) highlighted as one of its most important application domains.
EgoHRV is a method that estimates heart rate variability (HRV) and heart rate (HR) from the gaze cameras in egocentric headsets. It uses a 3D backbone and a low–high decomposition module to extract the blood volume pulse signal from gaze video, and aligns frequency‑domain representations of contact‑based and camera‑derived signals through cross‑domain pretraining. The approach achieves state‑of‑the‑art accuracy for HR and HRV estimation and, when integrated into EgoExo4D’s proficiency estimator, improves accuracy by 17.8%.
GazeFlow is a new framework for egocentric gaze prediction that models gaze as a joint distribution of temporal positions conditioned on both top‑down task cues and bottom‑up visual saliency. It employs conditional flow matching to iteratively transform Gaussian noise into realistic gaze trajectories, using a velocity field informed by video‑encoded visual features and global task queries. On standard benchmarks, GazeFlow outperforms existing methods on per‑frame metrics and produces trajectories that better reflect human gaze dynamics.
OpenVAM is a new framework for visual attention modeling that combines a dense saliency map with language‑based explanations. It uses a decoupled design: a visual pathway for precise localization and a vision‑language head that generates grounded what/why explanations. The method is trained in three stages to preserve localization while adding language grounding, and a scalable pipeline creates multi‑domain annotations for evaluation.
arXiv:2607.14935v2 Announce Type: replace Abstract: Recent advances in video understanding have spanned motion, long video, and streaming interaction, driving this field toward real-world application...
The paper addresses the problem of inaccurate latency reporting in browser-based webcam gaze trackers, which often timestamp samples at emission rather than capture time. It introduces a method that recovers a per-frame capture clock using the browser’s requestVideoFrameCallback API, enabling precise pairing of source frames with inference results or providing a verifiable lower bound when the engine does not expose its pipeline. An open TypeScript implementation and benchmark harness are released, tested on WebGazer and a FaceMesh+KRR pipeline.
arXiv:2609.00951v1 Announce Type: new Abstract: Collaborative perception shares information among multiple agents to obtain a comprehensive scene representation, enhancing the perceptual capability o...
arXiv:2603.26945v2 Announce Type: replace Abstract: Appearance-based gaze estimation (AGE) has achieved remarkable performance in constrained settings, yet we reveal a significant generalization gap...
arXiv:2608. 11367v1 Announce Type: cross Abstract: Estimating human gaze targets from images in-the-wild is an important and formidable task.
The paper introduces Salience-LLaVA, a vision‑language model that prioritizes scene elements based on their importance for low‑vision users. It presents three new salience‑aware datasets—Salience COCO, Salience Flickr, and Salience VizWiz—annotated with object‑level salience verified by low‑vision participants. The authors also propose the SCMI metric to evaluate caption ordering accuracy and demonstrate the system’s practicality by deploying it on assistive glasses.