arXiv Computation and Language

All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue

arXiv Computation and Language
Aug 25

ToSCA: Leveraging Hierarchical Reinforcement Learning on Temporal and Strategic Abstractions of Conversational Agents

arXiv:2608.21969v1 Announce Type: new Abstract: Humans have multiple levels of temporal abstractions on daily interaction and thinking, such as concept perception and strategic planning. Inspired by...

By Xiaoyu Wang, Qingqing Gu, Yue Zhao, Teng Chen, Yuqi Cao, Xiaokai Chen, Hongyan Li, Luo Ji
arXiv Computation and Language
Aug 28

Evaluating Language Models in Realistic Conversational Contexts

The paper introduces UPHELD, a large benchmark of human-to-human dialogues written by professional script writers, featuring realistic turn densities and over 36,000 per-turn human annotations. It evaluates existing automatic metrics and LLM-as-a-judge methods, finding them unreliable against expert human judgment. Using UPHELD, the authors develop a Mixture-of-Judges framework that improves correlation with human assessments by about 30%.

By Ilija Subasic, Andrew Rabinovich, Zhao Chen
arXiv Computation and Language
Sep 3

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding

TalkFa introduces a unified benchmark for Farsi dialogue generation and understanding, comprising three datasets: WIKI‑FADIAL (4.2K Wikipedia‑grounded dialogues), DAILYDIALOG‑FA (6.6K dialogues with dialogue‑act and emotion annotations), and PLAYDIAL‑FA (2.1K theatrical dialogues with sentiment labels). All dialogues are curated through multi‑stage review by native speakers, ensuring high quality. Experiments show that LoRA fine‑tuning improves generation performance with less data, while specific models excel on classification tasks, and human evaluation confirms the benchmark’s reliability.

By Neda Jamshidi, Kamyar Zeinalipour, Fahimeh Akbari, Monica Bianchini, Marco Maggini, Marco Gori