Is the Judge Its Twin? Post-training makes a model's writing more predictable but barely moves its taste, as a judge, toward predictable writing
Read the original on arXiv Machine Learning →The study investigates whether post‑training makes language models’ writing more predictable and whether this effect carries over when the same models act as judges. Using two 7‑billion‑parameter families (OLMo‑2 and Zephyr), the authors track changes from base models through supervised fine‑tuning (SFT) and direct preference optimization (DPO). They find that as writers, models produce increasingly familiar stories, but as judges, the models’ preference for predictable writing barely shifts, while they become more sensitive to story length, display order, and lose consistency on creative‑writing questions.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.