arXiv Machine Learning

Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations

The paper introduces a training‑free method for uncovering prompt‑conditional stylistic axes in large language models (LLMs). By repeatedly sampling completions of a single prompt at high temperature and applying Principal Component Analysis (PCA) to the pooled hidden activations, the authors automatically label the resulting axes using the extreme (pole) generations. Validation against 245 human‑elicited stylistic annotations shows that, for the Qwen‑3.5‑4B‑Instruct model, the top two axes align with human dimensions with 72.8% precision and 43.6% macro‑recall, and 75.6% of validity ratings confirm the axes’ polar generations, while other models exhibit varying degrees of discoverability.

arXiv AI
Sep 7

Can Activation Steering Capture Multidimensional Authorship Style?

The paper investigates whether activation steering can capture the complex, multidimensional nature of authorship style. By using structured contrastive prompting along rhetorically motivated dimensions, the authors construct rich style representations directly in activation space, revealing a shared authorship backbone with aspect‑specific residuals. They propose Aspect‑Aware Activation Steering (A3S), a training‑free framework that merges per‑aspect contrastive directions, employs interference‑aware aggregation, and tunes steering strength per instance, achieving better authorship style transfer and outperforming a trained baseline on out‑of‑domain benchmarks.

By Hieu Tran, Calvin Bao, Marine Carpuat
arXiv AI
Aug 20

Self- and Other-Labels Induce Bidirectional Bias in LLM Judges

The study investigates bias in large language model (LLM) judges by having ten LLMs evaluate narrative constraint selections rather than generated text. Results show that self-preference largely disappears under blind evaluation when quality and evaluator severity are controlled, but self- and other-labels alone shift scores bidirectionally when quality is matched. The authors conclude that authorship attribution drives evaluation bias and that open-ended, ground‑truth‑free tasks can effectively study LLM judge behavior.

By Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, Seohyon Jung
arXiv Computation and Language
Sep 2

Evaluating Style-Personalized Text Generation: Challenges and Directions

The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.

By Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar
arXiv Computation and Language
Sep 10

Deep and shallow biases in language models

The paper introduces a bias depth score to differentiate between stable model preferences (Deep biases) and prompt‑dependent responses (Shallow biases) in large language models. By analyzing 4,442 opinion prompts across four models, it finds that only about a quarter of concentrated preferences persist after scenario reframing, indicating that most are shallow. The study shows Deep biases are more often inherited from pretraining and harder to remove through fine‑tuning or prompt‑based debiasing, highlighting the need to distinguish learned biases from prompt artifacts.

By An Vo, Vy Tuong Dang, Khai-Nguyen Nguyen, Emilio Villa-Cueva, Thamar Solorio, Anh Totti Nguyen, Daeyoung Kim
arXiv AI
1d ago

Beyond Poetry: Can Large Language Models Generate Classical Arabic Maqamat?

arXiv:2609. 28245v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong performance in creative text generation, yet their ability to produce culturally grounded and stylistically constrained literary forms remains underexplored.

By AbdulRahman A. Morsy (Department of Computer Science, School of Engineering and Applied Sciences, George Washington University, Washington DC, United States), Aya Zirikly (Department of Computer Science, School of Engineering and Applied Sciences, George Washington University, Washington DC, United States, Center for Speech and Language Processing, Whiting School of Engineering, Johns Hopkins University, Baltimore MD, United States)
Hugging Face Trending Papers
Aug 3

Style Wins, Substance Loses: A Diagnosis of LLM-as-Judge in Idea Generation

However, whether these judges truly evaluate the scientific substance of ideas or are influenced by superficial stylistic presentation remains an open question. To address this question, we propose SciStyleBench, a unified three-component benchmark for diagnosing and mitigating stylistic bias in LLM-based idea evaluation: (i) First, SciStyleStage, a three-stage evaluation environment that applies controlled stylistic perturbations to fixed scientific content across three settings no context, fixed-domain context, and open-domain retrieval context, covering 600 scientific ideas and 15 style variants, with 9,000 evaluation instances per setting; (ii) Second, SciStyleMetrics, a set of quantitative measures, including Style Bias Index (SBI), Substance Recognition Rate (SRR), and Adversarial Win Rate (AWR), to characterize how stylistic variation affects scoring stability, substance discrimination, and ranking robustness; (iii) Third, SciStyleExtractor, a plug-and-play evaluation module that separates presentation style from scientific content by predicting style type and deviation before style-conditioned evaluation, enabling us to assess whether style awareness reduces stylistic bias.