Hugging Face Trending Papers

ReferClaw: Disambiguating User Intent for Egocentric Assistance

Read the original on Hugging Face Trending Papers →

ReferClaw is an agentic framework designed to improve egocentric AI assistants by treating question answering as an interactive process. It uses three cooperating agents to generate candidate interpretations, ask clarifying questions, and produce answers grounded in resolved intent and visual evidence. The system balances information gain against user effort, learns from interaction, and shows higher accuracy and joint answer‑grounding performance on augmented egocentric QA benchmarks, with zero‑shot transfer across datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI
Nov 6

ReferClaw: Disambiguating User Intent for Egocentric Assistance

ReferClaw is an agentic framework designed to improve egocentric AI assistants by treating question answering as an interactive process. It uses three cooperating agents to generate candidate interpretations, ask clarifying questions, and produce answers grounded in resolved intent and visual evidence. The system balances information gain against user effort, learns from interaction, and shows better accuracy and grounding on augmented egocentric QA benchmarks, including zero‑shot transfer across datasets.

By Hangyu Qin, Junbin Xiao, Angela Yao
arXiv AI
Sep 10

Companion-style QA Assistance in Ego-Vision

BuddyVQA is a new benchmark for companion‑style question answering on egocentric streaming video, comprising 21.6K questions tied to 6K highlight moments across 1,012 long first‑person videos. It emphasizes two often overlooked aspects of daily first‑person QA: ego‑deictic expressions and interactively chained questions, requiring models to resolve visual pronouns and infer user intent within a long‑form streaming context. The authors propose MyBuddy, a multimodal chain‑of‑thought QA assistant that uses a question filter and multi‑level memory to efficiently retrieve visual and QA information, achieving significant performance gains on BuddyVQA and generalizing to other streaming and common video QA benchmarks.

By Hangyu Qin, Junbin Xiao, Shenglang Zhang, Angela Yao
arXiv Machine Learning
Jul 7

Incentivizing Vision Language Models to Search for Long Video Question Answering

arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.

By Harsh Goel, S P Sharan, Sahil Shah, Minkyu Choi, Joungbin An, Kristen Grauman, Sandeep P. Chinchali