arXiv AI By Orian Dabod, Amir Cohen, Gabriel Stanovsky

When Reranking Hurts: Uncertainty-Based Gating for Few-Shot Reranking

Read the original on arXiv AI →

arXiv:2606. 31087v1 Announce Type: cross Abstract: Few-shot selection typically assumes that reranking retrieved examples always improves performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Aug 28

The Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions

The paper investigates the "score granularity gap" in black-box large language model (LLM) classifiers, asking how finely a confidence score can be thresholded for deployment. By comparing seven confidence construction methods across 25 model-dataset pairs, the authors find that single-shot verbalized confidence, when properly converted to a probability, ranks well but offers only a few distinct threshold values, limiting operational flexibility. The study also shows that multi-query aggregation can improve weak models but may harm strong ones, and provides concrete guidance for deployment trade-offs.

By Ao Sun, Tian Sun, Jiaxing Geng