One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs

The paper demonstrates that a single example from the BBQ fairness benchmark can dramatically improve a model’s performance, raising accuracy from 79.9% to 92.9% with Group Relative Policy Optimization and to 99.0% with one-shot in-context learning. This effect is consistent across different model families and is driven by the model’s reasoning traces, which adopt a category‑agnostic "missing evidence" pattern. The authors argue that BBQ-style multiple‑choice abstention tests capture only a single structural cue and therefore do not guarantee true fairness, calling for broader evaluation suites.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• reinforcement-learning
• benchmarks
• safety