arXiv Computation and Language September 3, 2026 By Smitha Muthya Sudheendra, Jaideep Srivastava

PragAlign: Feedback-Guided Pragmatic Alignment for Controlled Synthetic Dialogue Generation

Read the original on arXiv Computation and Language →

PragAlign is a feedback‑guided framework that improves synthetic dialogue generation by iteratively generating, evaluating, and revising conversations to meet specified service context, target intent, and target emotion. Using an LLM‑based evaluator that scores intent alignment, emotion alignment, coherence, fluency, and overall quality, PragAlign achieves a 99.50% acceptance rate on 800 dialogue specifications, outperforming one‑shot and repeated generation without feedback. Human evaluation confirms that intent expression and dialogue flow are reliably recognized, while emotion appropriateness remains more variable.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language

4d ago

CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamics

arXiv:2601.10085v3 Announce Type: replace Abstract: Conversational dynamics describe how rapport, emotion, and goals change over a dialogue. These changes are important in therapeutic conversations,...

By Viet Cuong Nguyen, Nhi Yen Nguyen, Kristin A. Candan, Mary Conlon, Vanessa Rumie, Kristen Risola, Michael L. Birnbaum, Munmun De Choudhury

llms agents benchmarks

arXiv AI

Jul 15

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

arXiv:2607. 12085v1 Announce Type: new Abstract: Evaluating retail conversational agents requires methods beyond lexical-overlap metrics to assess intent alignment, factuality, helpfulness, clarity, tone, and overall response quality.

By Niranjan Kumar M, Balaji Nagarajan, Karthik Nair, Faysal Satter, Nithin Surendran

llms agents benchmarks safety

arXiv AI

Jul 21

Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment

arXiv:2607. 17191v1 Announce Type: new Abstract: Human-like private chat requires more than fluent response generation: a system must preserve persona, relationship, memory, bounded knowledge, medium-specific timing, and a coherent multi-turn arc.

By Wentao Liu, Siyu Song, Xi Chen, Youjia Li, Xiaokun Wang, Min Ji, Ji Wang

reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 22

Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction

arXiv:2607. 18973v1 Announce Type: cross Abstract: Textual skills provide a lightweight way to improve frozen language-model agents, but their self-evolution normally requires a stable validation signal.

By ChaoJin Zhao, Xuan Jiang

agents safety

arXiv AI

Jul 3

HAL: Inducing Human-likeness in LLMs with Alignment

arXiv:2601. 02813v3 Announce Type: replace Abstract: Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize.

By Masum Hasan, Junjie Zhao, Ehsan Hoque

llms safety

arXiv Computation and Language

Aug 28

Evaluating Language Models in Realistic Conversational Contexts

The paper introduces UPHELD, a large benchmark of human-to-human dialogues written by professional script writers, featuring realistic turn densities and over 36,000 per-turn human annotations. It evaluates existing automatic metrics and LLM-as-a-judge methods, finding them unreliable against expert human judgment. Using UPHELD, the authors develop a Mixture-of-Judges framework that improves correlation with human assessments by about 30%.

By Ilija Subasic, Andrew Rabinovich, Zhao Chen

llms nlp benchmarks

One email a morning, machine-written

Related stories

CALM-IT: Enhancing Synthetic Motivational Interviewing Dialogue Quality with Conversational Dynamics

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment

Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction

HAL: Inducing Human-likeness in LLMs with Alignment

Evaluating Language Models in Realistic Conversational Contexts