arXiv AI September 10, 2026 By Hangyu Qin, Junbin Xiao, Shenglang Zhang, Angela Yao

Companion-style QA Assistance in Ego-Vision

BuddyVQA is a new benchmark for companion‑style question answering on egocentric streaming video, comprising 21.6K questions tied to 6K highlight moments across 1,012 long first‑person videos. It emphasizes two often overlooked aspects of daily first‑person QA: ego‑deictic expressions and interactively chained questions, requiring models to resolve visual pronouns and infer user intent within a long‑form streaming context. The authors propose MyBuddy, a multimodal chain‑of‑thought QA assistant that uses a question filter and multi‑level memory to efficiently retrieve visual and QA information, achieving significant performance gains on BuddyVQA and generalizing to other streaming and common video QA benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision

Oct 23

Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering

The paper introduces a retrieval-augmented online textual memory for episodic memory video question answering, focusing on long egocentric videos. It evaluates how different retrieval units, query formulations, context budgets, and reranking strategies affect multiple-choice answering on QAEgo4D-Closed, keeping the generated memory fixed. The best configuration, Parent, matches strong baselines while exposing only a small fraction of the textual memory, and the study shows that better temporal coverage does not always improve accuracy, highlighting the need for evidence-aware representations.

By Raffaele Cali, Giuseppe Lando, Rosario Forte, Antonino Furnari

llms rag nlp multimodal

arXiv AI

Jun 6

Personal AI Agent for Camera Roll VQA

arXiv:2606. 05275v1 Announce Type: cross Abstract: We study the personal camera roll visual question answering setting.

By Thao Nguyen, Krishna Kumar Singh, Donghyun Kim, Yong Jae Lee, Yuheng Li

agents nlp

arXiv Computer Vision

Oct 2

FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answering

FocusGraph is a framework for selecting keyframes in egocentric long‑video question answering. It uses a lightweight Scene‑Graph LLM Selector to identify query‑relevant clips from graph‑based captions, then extracts keyframes with Patch‑wise Sparse‑Flow Retention (PSFR) before feeding them to a multimodal large language model for answer generation. The method achieves state‑of‑the‑art performance on FindingDory and HourVideo while reducing question‑time inference cost.

By Tatiana Zemskova, Solomon Andryushenko, Ilya Obrubov, Viktoriia Khoruzhaia, Ekaterina Eroshenko, Ekaterina Derevyanka, Dmitry Yudin

llms agents nlp robotics multimodal benchmarks

arXiv Machine Learning

Aug 3

EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environment

arXiv:2604. 08342v2 Announce Type: replace Abstract: Long context egocentric video understanding has recently attracted significant research attention, with augmented reality (AR) highlighted as one of its most important application domains.

By Qiance Tang, Ziqi Wang, Jieyu Lin, Ziyun Li, Barbara De Salvo, Sai Qian Zhang

benchmarks

arXiv AI

Jun 30

Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction

arXiv:2606. 29445v1 Announce Type: cross Abstract: Video understanding is a fundamental capability for multimodal intelligence, and recent Multimodal Large Language Models (MLLMs) have achieved remarkable performance on Video Question Answering (VideoQA) benchmarks.

By Sunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang

llms agents nlp multimodal benchmarks

Hugging Face Trending Papers

Jul 13

Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos

When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive.

rag agents benchmarks safety