arXiv AI
Aug 24

TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

TLive-Omni is an omni‑modal understanding model designed for e‑commerce live streaming, integrating image, video, audio, and text inputs into a unified representation. It introduces Per‑vGrid for timestamped token organization, a three‑stage supervised training pipeline, and a Faithful‑RFT reinforcement fine‑tuning stage to enhance answer faithfulness and expression quality. The model is supported by a scenario‑oriented capability taxonomy and a compact data production engine that generates training signals for tasks such as speech recognition, product visual grounding, and omni‑modal QA, achieving strong performance on live‑commerce benchmarks and good generalization to general tasks.

By Yibo Hu, Yu Qian, Mao Gu, Yingfan Tao, Yuhao Chen, Yongdong Luo, Zhuoqun Liu, Meiguang Jin, Junfeng Ma
arXiv Computer Vision
Sep 18

Grounded Product Understanding in Livestream Videos

The paper introduces GPUB, a large-scale benchmark for grounded product understanding in e‑commerce livestream videos, featuring 3,000 livestreams, 31K fashion products, and multi‑moment temporal annotations. It defines three evaluation tasks, with the main task (GPrU) requiring simultaneous product identification and moment localization. Existing multimodal models perform poorly on GPrU, prompting the authors to develop UniPro, which improves performance by learning product‑aligned, temporally structured representations.

By Xinyu Zhang, Junjie Chen, Jiawei Ge, Qianlong Li, Libin Ma, Baokun Pan, Yahui Luo
arXiv AI
Oct 5

Beyond Single Videos: Benchmarking and Active Evidence Seeking for E-Commerce Cross-Video Reasoning

The paper introduces AdsCVR, a benchmark for e‑commerce cross‑video reasoning with 2,483 videos and 6,110 QA pairs across six reasoning dimensions. It proposes AdSeek, an agentic framework that actively selects visual and audio tools during multi‑turn exploration and uses an offline trajectory rectification mechanism to improve reinforcement learning. AdSeek achieves 74.30% accuracy on AdsCVR, outperforming its backbone by 27.90 percentage points and generalizes to the open‑domain CrossVid benchmark.

By Jinghan Zhao, Yiman Hu, Liang Wu, Jian Xu, Bo Zheng
arXiv AI
Jun 19

VCG: A Multimodal Retrieval Framework for E-Commerce Video Feeds under Extreme Cold-Start Conditions

arXiv:2606. 19627v1 Announce Type: cross Abstract: The digital commerce landscape is shifting from static, search-driven catalogs to dynamic, immersive video feeds.

By Katya Mirylenka, Egor Malykh, Mahdyar Ravanbakhsh, Michael Gygli, Marco-Andrea Buchmann, Andrew Dzhoha, Svitlana Borzenko, Francesca Catino, Mohamed Gaafar, Maarten Versteegh, Thomas Kober, Dario d'Andrea, Ellie Langhans