arXiv AI By Durga Prasad Maram, Dhruvin Gandhi, Zonghai Yao, Gayathri Akkinapalli, Franck Dernoncourt, Yu Wang, Ryan A. Rossi, Nesreen K. Ahmed

Iterative Critique-Refine Framework for Enhancing LLM Personalization

Read the original on arXiv AI →

The paper introduces PerFine, a training‑free critique‑refine framework that improves large language model personalization. In each iteration, a generator drafts text conditioned on a retrieved user profile, while a critic—also conditioned on the same profile—provides structured feedback on tone, vocabulary, sentence structure, and topicality. The generator revises the draft, and a knockout strategy selects the best version across iterations; additional inference‑time techniques such as Best‑of‑N and Topic Extraction further balance quality and efficiency. Experiments on Yelp, Goodreads, and Amazon datasets show PerFine consistently outperforms existing retrieval‑augmented methods like PGraphRAG, achieving G‑Eval gains of +8.5‑12% and steady improvements over 3‑5 refinement iterations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 2

Evaluating Style-Personalized Text Generation: Challenges and Directions

The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.

By Anubhav Jangra, Bahareh Sarrafzadeh, Silviu Cucerzan, Adrian de Wynter, Sujay Kumar Jauhar