arXiv AI

An Auditable Symbolic-RAG-Generative AI Architecture for Goal-Oriented Conversation Orchestration

The paper introduces a Symbolic-RAG-Generative architecture called GRACE for goal‑oriented conversational systems. GRACE transforms business intent into a fixed objective set and uses a constrained policy to update the conversation state based on visitor‑authored evidence while ensuring visitor utility. Evaluation on real‑estate and professional‑cleaning dialogues shows high accuracy in state transitions, evidence precision/recall, and monotonicity.

arXiv Computation and Language

Sep 3

A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models

The paper presents a tri‑agent framework for evaluating large language models’ question‑clarification abilities. It involves a Question Clarifying Agent that identifies ambiguities and asks follow‑up questions, a Respondent Agent that simulates human replies, and an Evaluator Agent that judges the dialogue using metrics such as ambiguity handling, question quality, dialogue efficiency, language appropriateness, and intent alignment. The authors illustrate the approach with synthetic supply‑chain data and discuss validating the evaluator against human judgments.

By Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra

llms agents benchmarks safety

arXiv AI

Jul 15

Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking

arXiv:2607. 12085v1 Announce Type: new Abstract: Evaluating retail conversational agents requires methods beyond lexical-overlap metrics to assess intent alignment, factuality, helpfulness, clarity, tone, and overall response quality.

By Niranjan Kumar M, Balaji Nagarajan, Karthik Nair, Faysal Satter, Nithin Surendran

llms agents benchmarks safety

arXiv Computation and Language

Sep 1

CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation

arXiv:2601.12208v2 Announce Type: replace Abstract: Evaluating conversational systems in multi-turn settings remains a fundamental challenge. Conventional pipelines typically rely on manually defined...

By Yunzhe Li, Richie Yueqi Feng, Tianxin Wei, Chin-Chia Hsu

Hugging Face Trending Papers

Sep 2

A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models

Large Language Models (LLMs) are increasingly deployed in interactive systems where understanding user intent precisely is paramount. A key capability for such systems is effective question clarificat...

llms agents benchmarks safety

arXiv AI

Jun 9

Goal-Oriented Reasoning for RAG-based Memory in Conversational Agentic LLM Systems

arXiv:2605. 12213v2 Announce Type: replace Abstract: LLM-based conversational AI agents struggle to maintain coherent behavior over long horizons due to limited context.

By Jiazhou Liang, Armin Toroghi, Yifan Simon Liu, Faeze Moradi Kalarde, Liam Gallagher, Scott Sanner

llms rag agents

arXiv AI

Aug 5

Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolution

arXiv:2608. 01366v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts.

By B. Sankar, Pawni Yadav, Srinidhi Ranjini Girish, Amogh A. S

llms rag agents

arXiv Machine Learning

Jun 26

GRAG: Generic Response-Augmented Generation Framework for Personalized Conversational Systems

arXiv:2606. 21097v2 Announce Type: replace-cross Abstract: Deploying highly capable personalized conversational agents in resource-constrained or privacy-sensitive environments remains a significant challenge.

By Junfeng Liu, Christopher T. Symons, Ranga Raju Vatsavai

llms agents fine-tuning benchmarks safety

arXiv AI

Jul 28

CallBench: A Benchmark for Dual-Goal Coordination in Phone Call Assistants

arXiv:2607. 22635v1 Announce Type: new Abstract: Target-oriented dialogue systems have demonstrated strong capabilities in completing user goals through interactive conversations.

By Xuzhao Geng, Haozhao Wang, Xuelian Li, Zhenyu Yang, Haonan Lu, Rui Zhang, Ruixuan Li

benchmarks safety

arXiv AI

Aug 13

Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversation

arXiv:2604. 03924v2 Announce Type: replace-cross Abstract: Goal-oriented conversational systems require making sequential decisions under uncertainty about the user's intent, where the algorithm must balance information acquisition and target commitment over multiple turns.

By Xinyi Ling, Ye Liu, Reza Averly, Xia Ning

llms benchmarks

arXiv AI

Aug 18

Intent-Driven Situation Tracking for User-Centric Multi-Turn Agents

arXiv:2608. 15755v1 Announce Type: new Abstract: User-centric multi-turn agents must act on an evolving task situation shaped by changing user intents, accumulated tool-grounded facts, missing information, and execution constraints.

By Meiling Tao, Yiling Tao, Peng Wang

llms agents benchmarks

arXiv AI

Aug 17

AdsWorldEngine: A Self-Evolving Conversational Advertising Agent through Orchestrator and Tool Coevolution

arXiv:2608. 13833v1 Announce Type: cross Abstract: Conversational advertising aims to deliver useful ads within multi-turn assistant interactions.

By Simiao Zuo, Chenhui Xu, Yimeng Jia, Qiang Lou, Jian Jiao, Denis Charles

agents reinforcement-learning fine-tuning

arXiv AI

Jul 10

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection

arXiv:2601. 02871v3 Announce Type: replace Abstract: Task-oriented proactive dialogue agents play a pivotal role in recruitment, particularly for steering conversations towards specific business outcomes, such as acquiring social-media contacts for private-channel conversion.

By Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao

agents reinforcement-learning fine-tuning

Related stories