← Back to all news
arXiv AI By Jingyu Tang, Xu Yang, Zhehao Zhang, Zheng Wang, Qi Cheng, Haozhe Zhao, Manling Li, Wenxuan Li, Xide Xia, Jiaxin Pei

Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?

Read the original on arXiv AI →

Realtime-Bench is a new benchmark that evaluates whether frontier models can handle real‑time interactive tasks where delayed actions directly impact success. The suite includes dynamic tasks that advance under different time‑clock regimes—Wall Clock, Paused Clock, and Token Clock—to isolate serving latency from a model’s temporal reasoning ability. Results show that even the best models achieve only 44% success under real‑time constraints, highlighting the challenge of balancing decision quality with timely execution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Sep 23

TimeInteract: Towards Real-Time Interactive Intelligence for Streaming Time Series

TimeInteract introduces a new regime called Time-Series Interaction, enabling models to continuously perceive incoming time-series data and user intent, decide when to respond, and keep processing new observations during response generation. The system employs a dual-view streaming encoder, a response control mechanism, and a decoupled inference pipeline to avoid blocking. Evaluated on the newly created StreamTSI-34K dataset, TimeInteract outperforms existing LLMs, VLMs, and TSLMs across four interaction levels, achieving significant gains in accuracy, response triggering, and inference speed.

By Sheng Pan, Yongli Gu, Yiqing Guo, Warren Jin, Bo Du, Shirui Pan, Ming Jin
llms
More like this →