We’ve applied reinforcement learning from human feedback to train language models that are better at summarization.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at OpenAI Blog.
The paper introduces PLUS, a framework that uses reinforcement learning to generate text-based summaries of individual users’ preferences, characteristics, and past conversations. These summaries condition a reward model, allowing it to predict personalized response preferences and improving reward accuracy by 11–77 % over the standard Bradley‑Terry model. PLUS demonstrates robust performance with new users and topics, achieves a 25 % improvement over existing personalized RLHF techniques, and enables zero‑shot personalization for state‑of‑the‑art models like GPT‑4.
arXiv:2606. 03867v1 Announce Type: cross Abstract: Multi-Document Summarization (MDS) plays a critical role in distilling essential information from collections of textual data.
We trained “critique-writing” models to describe flaws in summaries. Human evaluators find flaws in summaries much more often when shown our model’s critiques.
We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.
arXiv:2606. 08000v1 Announce Type: cross Abstract: The progress of large language models (LLMs) has fueled claims that model-generated summaries rival or even surpass human-written references, raising questions about whether summarization remains an open research problem.