The paper introduces TextQ‑German, a dataset suite for evaluating German natural language generation (NLG) from a Quality of Experience (QoE) perspective, covering tasks such as summarization and machine translation. Human ratings collected via crowdsourcing identify perceptual quality dimensions, and the authors develop automatic QoE prediction models—including transformer‑based, linguistic feature‑based, and hybrid approaches—showing that hybrid models outperform pure transformers and that linguistic features alone can rival fine‑tuned language models. The dataset is further enriched with large language model (LLM) outputs annotated with overall QoE scores, and validation on held‑out data demonstrates generalization to unseen data.
"whyItMatters":"The resource provides a publicly accessible benchmark and baseline models for human‑centered NLG evaluation, enabling the development of systems that better align with human quality perception."
arXiv:2507.01594v2 Announce Type: replace
Abstract: Task-oriented dialogue (ToD) systems aim to help users accomplish goals through natural language interaction. Beyond task success, effective ToD sy...
By Shutong Feng, Hsien-chin Lin, Nurul Lubis, Carel van Niekerk, Michael Heck, Benjamin Ruppik, Renato Vukovic, Milica Ga\v{s}i\'c
arXiv:2608. 13606v1 Announce Type: new Abstract: The next generation of AI agents is increasingly moving beyond systems that answer isolated questions toward persistent personal assistants that can understand, remember, and continuously learn from users' experiences.
By Xinle Deng, Yida Xue, Xiangyuan Ru, Haoming Xu, Shuofei Qiao, Mengru Wang, Yijun Chen, Buqiang Xu, Chen Jiang, Yuchen Eleanor Jiang, Lizhong Wang, Jianfeng Wang, Li Zeng, Haofen Wang, Guilin Qi, Huajun Chen, Ningyu Zhang
arXiv:2606. 06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data.
By Lechen Zhang, Jiarui Liu, Tal August
arXiv:2609.17010v1 Announce Type: new
Abstract: Lifelong conversational agents rely on memory systems to maintain deep, context-aware interactions with users. However, existing explicit textual memor...
By Cai Ke, Xin Liu, Han Zhang, Jiangyue Yan, Zike Yuan, Ling Deng, Yue Yu, Hui Wang, Ruifeng Xu
arXiv:2601. 06862v2 Announce Type: replace-cross Abstract: The quality of the user experience has become one of the most important aspects in todays world, as it directly influences individuals willingness to continue using or abandon a product or service.
By Michael Sidorov, Ofer Hadar
arXiv:2604. 25834v2 Announce Type: replace Abstract: With the rapid development of the Internet, users have increasingly higher expectations for the recommendation accuracy of online content consumption platforms.
By Wenhao Li, Zihan Lin, Zhengxiao Guo, Jie Zhou, Shukai Liu, Yongqi Liu, Chuan Luo, Chaoyi Ma, Ruiming Tang, Han Li
arXiv:2606. 24042v1 Announce Type: new Abstract: Recommender systems often induce filter bubbles and semantic homogenization by monolithically optimizing for immediate user engagement.
By Cl\'audio L\'ucio Do Val Lopes, Lucca Machado da Silva, Andr\'e de Oliveira Brand\~ao
arXiv:2609.20001v1 Announce Type: new
Abstract: Automated video interview assessment integrates verbal content, acoustic delivery, and visual behavior, yet numerical predictions alone provide limited...
By Haoshen Wang, Dongbo Che, Zeyi Xie, Yuanjie Du, Shicheng Hua, Xingyu Wang
User interface (UI) and user experience (UX) evaluation is central to product development, yet reliable feedback still relies on recruiting human participants or running online A/B tests, making early-stage iteration slow and costly. In light of this, recent work has explored Multimodal Large Language Models as proxy evaluators.
arXiv:2608. 11604v1 Announce Type: new Abstract: Large language model-based shopping agents are increasingly deployed in real-world e-commerce platforms, generating massive amounts of user interaction logs that provide valuable supervision for improving these agents.
By Haobo Zhang, Kelong Mao, Sulong Xu, Simiu Gu, Zhicheng Dou
arXiv:2606. 03635v1 Announce Type: cross Abstract: Understanding short online videos involves more than identifying visible objects and actions; video makers often include an underlying message or purpose in the clip.
By Issar Tzachor, Michael Green, Rami Ben-Ari