The paper investigates whether activation steering can capture the complex, multidimensional nature of authorship style. By using structured contrastive prompting along rhetorically motivated dimensions, the authors construct rich style representations directly in activation space, revealing a shared authorship backbone with aspect‑specific residuals. They propose Aspect‑Aware Activation Steering (A3S), a training‑free framework that merges per‑aspect contrastive directions, employs interference‑aware aggregation, and tunes steering strength per instance, achieving better authorship style transfer and outperforming a trained baseline on out‑of‑domain benchmarks.
By Hieu Tran, Calvin Bao, Marine Carpuat
The study investigates bias in large language model (LLM) judges by having ten LLMs evaluate narrative constraint selections rather than generated text. Results show that self-preference largely disappears under blind evaluation when quality and evaluator severity are controlled, but self- and other-labels alone shift scores bidirectionally when quality is matched. The authors conclude that authorship attribution drives evaluation bias and that open-ended, ground‑truth‑free tasks can effectively study LLM judge behavior.
By Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, Seohyon Jung
The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.
By Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar
arXiv:2310.00436v2 Announce Type: replace
Abstract: Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This surv...
By Haining Wang
arXiv:2608. 01666v2 Announce Type: replace-cross Abstract: However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open question.
By Fengxian Ji, Yuke Li, Jingpu Yang, Juanfan Wu, Fan Zhang, Zhexuan Cui, Yu Xie, Min Peng, Qianqian Xie, Xiuying Chen, Zhuohan Xie
The paper introduces a bias depth score to differentiate between stable model preferences (Deep biases) and prompt‑dependent responses (Shallow biases) in large language models. By analyzing 4,442 opinion prompts across four models, it finds that only about a quarter of concentrated preferences persist after scenario reframing, indicating that most are shallow. The study shows Deep biases are more often inherited from pretraining and harder to remove through fine‑tuning or prompt‑based debiasing, highlighting the need to distinguish learned biases from prompt artifacts.
By An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen, Emilio Villa-Cueva, Thamar Solorio, Anh Totti Nguyen, Daeyoung Kim