arXiv AI By Chen Li, Dimitrios Chrysostomou

IRWOZ 2.0: A Large Language Model-driven Dialogue Dataset for Industrial Robot Conversations

Read the original on arXiv AI →

IRWOZ 2.0 is a refined dialogue dataset for industrial human‑robot interaction, expanding to 390 dialogues across four domains—Assembly, Delivery, Position, and Relocation. The dataset was improved using large language models (Mistral and Claude‑3.5) for generation and quality refinement, including manual corrections and automated typo removal. Benchmark tests show a substantial boost in dialogue state‑tracking performance, with GPT‑2’s BLEU‑4 score rising from 0.1651 to 0.5604 compared to the original IRWOZ.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 3

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding

TalkFa introduces a unified benchmark for Farsi dialogue generation and understanding, comprising three datasets: WIKI‑FADIAL (4.2K Wikipedia‑grounded dialogues), DAILYDIALOG‑FA (6.6K dialogues with dialogue‑act and emotion annotations), and PLAYDIAL‑FA (2.1K theatrical dialogues with sentiment labels). All dialogues are curated through multi‑stage review by native speakers, ensuring high quality. Experiments show that LoRA fine‑tuning improves generation performance with less data, while specific models excel on classification tasks, and human evaluation confirms the benchmark’s reliability.

By Neda Jamshidi, Kamyar Zeinalipour, Fahimeh Akbari, Monica Bianchini, Marco Maggini, Marco Gori
arXiv Computation and Language
Sep 11

Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue

The paper introduces Inverse Turing Bench, a benchmark designed to assess how well language models can distinguish between human-only and human-AI dialogues in multi-turn text. It provides paired dialogue transcripts and evaluates models on correctly identifying the type of conversation. Preliminary results show GPTZero, Claude Opus-4.6, and GPT-5.5 achieving the highest accuracies of 89.41%, 77.92%, and 75.94% respectively, highlighting both the strengths and limitations of statistical versus semantic detection approaches.

By William Hager, Ishika Rathi, Masum Hasan, Cameron Jones
arXiv Computation and Language
Sep 4

Decoupling Turn-Taking from Semantics: A Decoupled Data Approach for Finite-State-Machine-Based Full-Duplex Dialogue

The paper introduces a decoupled data approach for the Neural Finite State Machine (NFSM) framework to improve full‑duplex dialogue. It serializes real human‑human spoken dialogues into FSM tapes using a rule‑based event‑guided transformation, while shaping semantics through human‑agent text dialogues. A Source‑Aware Calibrated (SAC) loss is proposed to balance state‑transition token distribution and align each data source with its strongest supervisory signal, leading to better turn‑taking performance without sacrificing semantic quality.

By Yihang Li, Chenhui Chu