arXiv:2606. 31087v1 Announce Type: cross Abstract: Few-shot selection typically assumes that reranking retrieved examples always improves performance.
arXiv:2506.17251v3 Announce Type: replace-cross Abstract: Although large language models (LLMs) have achieved remarkable performance, the inherent stochasticity of their reasoning processes and varyi...
arXiv:2606. 01070v1 Announce Type: cross Abstract: Dense retrievers excel at first-stage candidate generation but lack effective reranking in zero-resource settings.
arXiv:2609.00588v1 Announce Type: new Abstract: Reranking methods, such as Minimum Bayes Risk (MBR) decoding and Quality Estimation (QE) reranking, are widely used in modern neural machine translatio...
arXiv:2609.37345v1 Announce Type: new Abstract: Streaming video understanding requires Video Large Language Models (Video-LLMs) to reason over continuous visual streams under causal constraints. As t...
arXiv:2606. 27997v1 Announce Type: new Abstract: Benchmarks of machine learning models often include many datasets, making evaluation expensive.
The paper investigates the "score granularity gap" in black-box large language model (LLM) classifiers, asking how finely a confidence score can be thresholded for deployment. By comparing seven confidence construction methods across 25 model-dataset pairs, the authors find that single-shot verbalized confidence, when properly converted to a probability, ranks well but offers only a few distinct threshold values, limiting operational flexibility. The study also shows that multi-query aggregation can improve weak models but may harm strong ones, and provides concrete guidance for deployment trade-offs.
arXiv:2606. 15216v1 Announce Type: cross Abstract: Diversity plays a critical role in data selection, improving performance under fixed data budgets by reducing redundancy and repetition.
arXiv:2606. 05308v1 Announce Type: new Abstract: With PRECISE, we extended Prediction-Powered Inference to produce bias-corrected estimates of ranking evaluation metrics by combining a small human-labeled set with a large LLM-judged set.
The study evaluates large language models (LLMs) as rerankers in conversational movie recommendation, comparing proprietary, open-weight, and fine-tuned LLMs against collaborative-filtering and sequential baselines on the ReDial benchmark. Results show that the best proprietary LLM achieves an NDCG@10 of 0.1497 with a shared semantic candidate pool, outperforming non-LLM baselines, while open-weight LLMs do not surpass a tuned shallow autoencoder under the same protocol. The analysis also highlights that reranker performance is highly sensitive to candidate generation, pool size, scoring policy, and decoding temperature, suggesting these factors should be reported as standard evaluation fields.
arXiv:2607. 16259v1 Announce Type: new Abstract: Pretrained models are typically ranked on multi-task leaderboards to assess their effectiveness across diverse tasks.
arXiv:2601.13885v2 Announce Type: replace-cross Abstract: Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluatio...
arXiv:2605. 04495v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on evidence ranking to determine what information is exposed to the generator, yet existing retrieval and reranking methods primarily estimate query--document relevance.