The paper introduces a register-aware framework to evaluate how human-like large language models (LLMs) are, focusing on linguistic feature distributions rather than factual correctness. It uses Maximum Mean Discrepancy (MMD) and 67 Biber lexico‑grammatical features to compare LLM‑generated texts with human reference corpora across different registers. Experiments on seven instruction‑tuned, open‑source models across five English datasets show that all LLMs deviate from human baselines, with closeness to human language varying by register and not by model size.
By Bj\"orn Nieth, Marianna Gracheva, Michaela Mahlberg, Bjoern Eskofier, Emmanuelle Salin
The paper introduces a reference-based framework to analyze coherence and diversity in open-ended text generation. It evaluates these properties by aligning them with human trajectories, comparing them to human continuations, and estimating their likelihood under a human reference distribution. Experiments show that diversity alignment and mean-based comparisons correlate with human quality ratings, while reference likelihood also associates positively, though results vary by configuration.
By Esteban Garc\'es Arias
arXiv:2606. 17350v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled the generation of high-quality prose, yet the question of whether these models are capable of generating diverse outputs remains contested.
By Thennal DK, Hans Ole Hatzel
arXiv:2608. 01423v1 Announce Type: cross Abstract: Reference-based text evaluation metrics, which are widely used to assess natural language generation systems, score a candidate response by comparing it with a reference response.
By Shengwei Xu, Yuxuan Lu, Yifan Wu, Jason Hartline, Grant Schoenebeck
Reference-based text evaluation metrics, which are widely used to assess natural language generation systems, score a candidate response by comparing it with a reference response. The reliability of an evaluation metric is usually judged by its statistical correlation with human ratings.
The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.
By Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar
The paper introduces STAR, a metric that measures sentence-level alignment between source and target documents in document-to-document machine translation. Using STAR, the authors develop StarPO, a preference‑optimization framework that ranks translation hypotheses by structural quality and applies a dynamic alignment mask to focus learning on misaligned segments. Experiments on news and literary data show that StarPO improves both translation quality and structural integrity, enabling small models to outperform large proprietary systems such as GPT‑4o while remaining more token‑efficient.
By Yichen Dong, Hao Wang, Junhui Li, Linlong Xu, Longyue Wang, Weihua Luo
arXiv:2609.07798v1 Announce Type: cross
Abstract: Natural Language Processing (NLP) in the climate domain requires models to process heterogeneous text sources, including scientific literature, polic...
By Yongan Yu, Shantam Raj, Jingwei Ni, Ario Saeid Vaghefi, Dominik Stammbach, Markus Leippold
arXiv:2606. 03165v1 Announce Type: cross Abstract: The language used by digital chat assistants such as ChatGPT can diverge from human expectations (misalignment).
By Thomas Stephan Juzek, Xiaoyang Ming, Jose A. Hernandez
arXiv:2609.34240v2 Announce Type: replace-cross
Abstract: Existing open-ended generation metrics measure likelihood, lexical diversity, or distributional similarity in generic representation space, y...
By Jinnuo Liu, Junhao Zhu, Weifeng Jiang, Haoming Liu, Hongyi Wen
The paper investigates cross‑lingual transfer for sequential sentence classification (SSC) in research papers, focusing on 13 non‑English languages. Experiments show that linguistic proximity does not reliably predict transfer success, whereas structural similarity in rhetorical organization—particularly label distribution similarity—correlates positively with performance. The authors introduce three generative‑model methods that exploit structural cues, achieving parity with strong encoder baselines on‑domain and outperforming them when transferring to unseen languages.
By Kazuhiro Yamauchi, Marie Katsurai
arXiv:2405.06818v2 Announce Type: replace
Abstract: Natural Language Processing (NLP) for Ghana's 73 living indigenous languages remains deeply fragmented, under-resourced, and heavily skewed toward...
By Sheriff Issaka, Erick Rosas Gonzalez, Colene Agbo, Evans Kofi Agyei, Shruti Tyagi, John Emeka Eze, Enock Appiah Tieku, Junlin Fang, Thanh Do Nguyen, Juliet Arthur, Zhaoyi Zhang, Mihir Heda, Keyi Wang, Yinka Ajibola, Rebecca Akpanglo-Nartey, Frank Lawrence Nii Adoquaye Acquaye, Dennis Owusu, Jerry John Kponyo, Stephen Moore, Isaac Wiafe, Sean Du