Pairwise AUC Optimization Needs Corrective Power: A Unified View

The paper investigates why direct pairwise AUC surrogate training, though aligned with the ranking metric, can fail to provide corrective updates during stochastic training. It introduces the concept of Corrective Power, showing that in certain regions severe ranking errors persist without generating strong gradient signals, a phenomenon linked to degeneracies in the U‑statistics of pairwise gradients. The authors explain empirical observations—such as the effectiveness of squared‑hinge loss, the benefit of cross‑entropy warm‑up, and the advantage of parameter‑efficient fine‑tuning—through this theoretical lens, and validate their claims with experiments on eight computer‑vision and eight NLP tasks using various models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• nlp
• fine-tuning
• benchmarks
• safety