arXiv AI
Jul 1

STEB: Style Text Embedding Benchmark

arXiv:2606. 31741v1 Announce Type: cross Abstract: While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets.

By Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti
arXiv AI
Sep 7

Can Activation Steering Capture Multidimensional Authorship Style?

The paper investigates whether activation steering can capture the complex, multidimensional nature of authorship style. By using structured contrastive prompting along rhetorically motivated dimensions, the authors construct rich style representations directly in activation space, revealing a shared authorship backbone with aspect‑specific residuals. They propose Aspect‑Aware Activation Steering (A3S), a training‑free framework that merges per‑aspect contrastive directions, employs interference‑aware aggregation, and tunes steering strength per instance, achieving better authorship style transfer and outperforming a trained baseline on out‑of‑domain benchmarks.

By Hieu Tran, Calvin Bao, Marine Carpuat
arXiv Computation and Language
Sep 2

Evaluating Style-Personalized Text Generation: Challenges and Directions

The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.

By Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar