← Back to all news
arXiv Computation and Language

Towards Proactive Detection of User-Side Implicit Conflicts in Human-LLM Dialogue

The paper introduces UC-Bench, a human‑annotated benchmark for detecting user‑side implicit conflicts in Human‑LLM dialogue, a problem largely overlooked compared to LLM‑side conflicts. Experiments show current LLMs struggle with these conflicts, especially when they stem from implicit incompatibilities in dialogue history. To address this, the authors propose SynUC, a constraint‑guided data synthesis method that generates a new training set, UC‑Data, which improves performance of lightweight LLMs on UC‑Bench compared to larger general‑purpose models and existing synthesis approaches.

Related stories

arXiv AI
Aug 28

When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue

The paper introduces ContraTalk, a benchmark that tests whether dialogue models truly use acoustic cues or rely on transcript shortcuts. It formalizes cross‑modal disagreement, creates conflict and consistent QA examples, and proposes an Audio Twin representation to expose acoustic evidence to models. Experiments show that while text‑only LLMs perform well on consistent cases, they falter on conflict cases, and AudioLLMs only partially mitigate this issue.

By Yen-Ju Lu, Yuzhe Wang, Yaohan Guan, Xiluo He, Jiarui Hai, Mingrui Liang, Kaavya Chaparala, Thomas Thebaud, Laureano Moro-Velazquez, Najim Dehak, Jesus Villalba
llmsagentsmultimodalbenchmarkssafety
More like this →
arXiv Computation and Language
Sep 4

Decoupling Turn-Taking from Semantics: A Decoupled Data Approach for Finite-State-Machine-Based Full-Duplex Dialogue

The paper introduces a decoupled data approach for the Neural Finite State Machine (NFSM) framework to improve full‑duplex dialogue. It serializes real human‑human spoken dialogues into FSM tapes using a rule‑based event‑guided transformation, while shaping semantics through human‑agent text dialogues. A Source‑Aware Calibrated (SAC) loss is proposed to balance state‑transition token distribution and align each data source with its strongest supervisory signal, leading to better turn‑taking performance without sacrificing semantic quality.

By Yihang Li, Chenhui Chu
llmsagentsfine-tuning
More like this →
arXiv Computation and Language
Sep 11

Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction

The paper introduces CoCoEval, a framework for evaluating large language model (LLM)–simulated conversations by detecting 10 types of inconsistent and uncollaborative behaviors at the turn level. Using CoCoEval, the authors compare human conversations with those generated by GPT‑4.1, GPT‑5.1, and Claude Opus 4, finding that LLMs produce far fewer such behaviors under vanilla prompting and that prompt engineering or fine‑tuning often over‑produces specific behaviors. The study highlights gaps between human and LLM‑simulated interactions that conventional Likert‑scale evaluations miss, raising concerns about using LLMs as proxies for human social interaction.

By Ryo Kamoi, Ameya Godbole, Binglin Zhou, Xiaoxin Lu, Longqi Yang, Rui Zhang, Mengting Wan, Pei Zhou
llmsfine-tuningbenchmarks
More like this →