arXiv AI By Qi Liu, Xiaoyang Yuan, Yubin Ruan, Zhuomeng Zhang, Wenjin Wang, Di Wu, Mingye Xu, Xinyi Mou, Xingxi Yin, Ke Feng, Zixun Sun

State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI
Sep 17

When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI

The paper evaluates three approaches for emotion recognition in conversation— a low‑cost stacked ensemble, an off‑the‑shelf LLM prompt, and a confidence‑gated hybrid that escalates only uncertain ensemble predictions to the LLM. Across three datasets (IEMOCAP, MELD, CMU‑MOSI), the hybrid consistently outperforms each pure system, achieving higher weighted F1 scores while routing most traffic through the inexpensive ensemble. This results in significant cost savings (≈$10‑85 per million utterances) and provides an interpretable escalation signal tied to emotion or sentiment shifts.

By Sai Babu Udayagiri, Arjun Chouhan, Ravisekhar Kanagala, Trishala Pavagada
arXiv Machine Learning
Sep 14

GAUGE: When Not to Trust LLM-as-a-Judge in User-Simulated Evaluation of Task-Oriented Agents

GAUGE is a new offline protocol that evaluates whether the common practice of using an LLM-as-a-judge to rank task‑oriented agents actually aligns with a verifiable reward. Across 25 agents from six providers on two benchmarks, GAUGE finds that user satisfaction scores are largely uncorrelated with task success, and that the judge’s ranking loses precision when agents are closely matched in performance. The study highlights a gap between ranking validity and construct validity in current evaluation practices.

By Umesh Bodhwani, Thanh Tran, Kai Wei