arXiv Machine Learning September 18, 2026 By Gunwoo Lee, Changmin Sung, Sang-Hwan Gwak, Ina Kim, Ji-Young Choi, Kyong-Ha Lee

Intrinsic Sequence-Likelihood Confidence in Retrieval-Dominated Extractive QA: Two Pre-Specified Negatives, and What They Do and Do Not Attribute

Read the original on arXiv Machine Learning →

The paper investigates whether confidence signals from fine‑tuned large language models can improve extractive question answering that relies heavily on retrieval. Experiments on four 7‑9B model families show that retrieval alone recovers 92–99.8% of the best possible accuracy, leaving little room for confidence‑based routing or adaptation to help. The sequence‑likelihood confidence metric, even after recalibration or temperature scaling, fails to provide a statistically significant benefit across different correctness criteria and answer lengths, and the study ultimately offers a set of pre‑specified negatives with explicit dependencies as its main contribution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 7

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

arXiv:2607. 04223v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) reduces but does not eliminate hallucination, and existing detectors return a single answer-level score that does not indicate which sentence is unsupported, or why.

By Mohamed Aly Bouke

rag nlp benchmarks safety

arXiv AI

Jul 29

Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs

arXiv:2607. 25600v1 Announce Type: cross Abstract: Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation.

By Chandan Kumar Sah, Xiaoli Lian, Li Zhang

llms rag nlp benchmarks

Hugging Face Trending Papers

Jul 28

Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs

Retrieval-augmented generation improves knowledge-intensive question answering, but indiscriminate retrieval can introduce irrelevant evidence and unnecessary computation. We investigate whether verbalized confidence from black-box language models can serve as an actionable signal for retrieval routing.

llms rag nlp benchmarks

arXiv AI

Aug 10

Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration

arXiv:2605. 27752v3 Announce Type: replace Abstract: Is verbalized confidence better calibrated than token likelihood?

By Hankyeol Kim, Pilsung Kang

llms

arXiv AI

Sep 1

Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs

The paper evaluates how modern large language models use internal web search to answer factual questions. Using 783 static queries and 288 dynamic queries, the authors find that enabling retrieval improves accuracy on static questions but hurts confidence calibration. On dynamic queries, models often retrieve but still achieve less than 70% accuracy, mainly due to poor query formulation and source selection, indicating that internal web search works better as a quick verification tool than a full information‑retrieval system.

By Sahil Kale

llms

arXiv AI

Jun 30

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering

arXiv:2606. 29090v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) has become the standard way to ground large language models in external knowledge, yet most systems retrieve a fixed number of passages for every question regardless of its difficulty.

By Ansh Kamthan

llms rag nlp

One email a morning, machine-written

Related stories

Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs

Beyond Self-Knowledge: Propagating Uncertainty Across Reasoning and Retrieval in LLMs

Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration

Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs

AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering