The paper introduces TextQ‑German, a dataset suite for evaluating German natural language generation (NLG) from a Quality of Experience (QoE) perspective, covering tasks such as summarization and machine translation. Human ratings collected via crowdsourcing identify perceptual quality dimensions, and the authors develop automatic QoE prediction models—including transformer‑based, linguistic feature‑based, and hybrid approaches—showing that hybrid models outperform pure transformers and that linguistic features alone can rival fine‑tuned language models. The dataset is further enriched with large language model (LLM) outputs annotated with overall QoE scores, and validation on held‑out data demonstrates generalization to unseen data.
"whyItMatters":"The resource provides a publicly accessible benchmark and baseline models for human‑centered NLG evaluation, enabling the development of systems that better align with human quality perception."
arXiv:2507.01594v2 Announce Type: replace
Abstract: Task-oriented dialogue (ToD) systems aim to help users accomplish goals through natural language interaction. Beyond task success, effective ToD sy...
By Shutong Feng, Hsien-chin Lin, Nurul Lubis, Carel van Niekerk, Michael Heck, Benjamin Ruppik, Renato Vukovic, Milica Ga\v{s}i\'c
arXiv:2608. 13606v1 Announce Type: new Abstract: The next generation of AI agents is increasingly moving beyond systems that answer isolated questions toward persistent personal assistants that can understand, remember, and continuously learn from users' experiences.
By Xinle Deng, Yida Xue, Xiangyuan Ru, Haoming Xu, Shuofei Qiao, Mengru Wang, Yijun Chen, Buqiang Xu, Chen Jiang, Yuchen Eleanor Jiang, Lizhong Wang, Jianfeng Wang, Li Zeng, Haofen Wang, Guilin Qi, Huajun Chen, Ningyu Zhang
arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.
By Lechen Zhang, Jiarui Liu, Tal August
arXiv:2609.17010v1 Announce Type: new
Abstract: Lifelong conversational agents rely on memory systems to maintain deep, context-aware interactions with users. However, existing explicit textual memor...
By Cai Ke, Xin Liu, Han Zhang, Jiangyue Yan, Zike Yuan, Ling Deng, Yue Yu, Hui Wang, Ruifeng Xu
arXiv:2601. 06862v2 Announce Type: replace-cross Abstract: The quality of the user experience has become one of the most important aspects in todays world, as it directly influences individuals willingness to continue using or abandon a product or service.
By Michael Sidorov, Ofer Hadar