arXiv:2609. 27303v1 Announce Type: new Abstract: Livestreams are long-lasting interactive environments where audiovisual content, viewer activity, host behavior, and platform signals evolve together, creating assistance needs that emerge from the stream itself.
arXiv:2606. 17798v1 Announce Type: cross Abstract: Despite the remarkable progress of Video Large Language Models (Video-LLMs), current online architectures still struggle to simultaneously process continuous video streams, decide autonomously when to respond, and preserve long-horizon contextual memory.
arXiv:2606. 05121v1 Announce Type: cross Abstract: Audio is an inherently interactive modality, yet today's Large Audio Language Models (LALMs) are offline, and streaming audio models each handle only a single task such as streaming ASR or voice chatting.
arXiv:2606. 06991v1 Announce Type: cross Abstract: Online Video Large Language Models (Video-LLMs) have advanced toward seamless human-AI interaction through frame-by-frame processing and proactive responding.
Natural interaction in digital and physical environments requires continuous perception and timely responses. Spoken dialogue relies on acoustic and linguistic cues, while video interaction also requi...
arXiv:2609.13814v1 Announce Type: new Abstract: Natural interaction in digital and physical environments requires continuous perception and timely responses. Spoken dialogue relies on acoustic and li...
The paper introduces a real‑time framework for generating co‑speech gestures for digital humans, coupling a streaming speech response module with a causal multimodal autoregressive gesture generator that uses only current speech and motion history. It also presents an offline data synthesis pipeline for virtual companion dialogues and a self‑evolving training loop that incorporates user feedback to continually adapt the model. Experiments show the system achieves a better latency‑quality trade‑off, stronger speech‑motion synchronization, and higher user preference than existing baselines.
arXiv:2608. 08469v1 Announce Type: new Abstract: Existing streaming multimodal models process observations incrementally but still follow a turn-based prefill-then-decode pattern, making them non-duplex: new observations cannot naturally enter an active generation stream.
TLive-Omni is an omni‑modal understanding model designed for e‑commerce live streaming, integrating image, video, audio, and text inputs into a unified representation. It introduces Per‑vGrid for timestamped token organization, a three‑stage supervised training pipeline, and a Faithful‑RFT reinforcement fine‑tuning stage to enhance answer faithfulness and expression quality. The model is supported by a scenario‑oriented capability taxonomy and a compact data production engine that generates training signals for tasks such as speech recognition, product visual grounding, and omni‑modal QA, achieving strong performance on live‑commerce benchmarks and good generalization to general tasks.
TimeInteract introduces a new regime called Time-Series Interaction, enabling models to continuously perceive incoming time-series data and user intent, decide when to respond, and keep processing new observations during response generation. The system employs a dual-view streaming encoder, a response control mechanism, and a decoupled inference pipeline to avoid blocking. Evaluated on the newly created StreamTSI-34K dataset, TimeInteract outperforms existing LLMs, VLMs, and TSLMs across four interaction levels, achieving significant gains in accuracy, response triggering, and inference speed.
TEMPO is a unified model that adds temporally‑grounded capabilities to large audio‑language models, enabling timestamping of events, speakers, and sounds in audio, speech, and music. It introduces a supervised fine‑tuning stage featuring atomic timestamp tokens, a time‑aware projector with sinusoidal encodings, and a distance‑aware Gaussian loss, trained via a synthetic‑to‑real curriculum. Additionally, TEMPO employs reinforcement learning (GRPO) as a refinement step, and achieves state‑of‑the‑art performance on a benchmark of 10K samples across five timestamping tasks, surpassing Audio Flamingo Next and Qwen3‑Omni.
arXiv:2609.05837v1 Announce Type: new Abstract: LLM-based agents are increasingly deployed in real-world applications through tool-use APIs, yet training them for specific environments remains fundam...
arXiv:2607. 11523v1 Announce Type: cross Abstract: When should an intelligent assistant speak up without being asked?