MAPLE is a framework that characterizes how Knowledge Extraction Enhancement (KEE) techniques influence parametric knowledge extraction in large language models. It introduces diagnostic metrics that separate effects on surface answers, probability distributions, and beyond-sampling recovery, measuring answer recovery, disruption, token rank shifts, and missed answer recovery. The authors evaluate six KEE methods across three model families—OLMo2, Qwen2.5, and Llama3—showing distinct effects and trade‑offs, and argue that MAPLE provides deeper insight than aggregate accuracy alone.
arXiv:2608. 20281v1 Announce Type: cross Abstract: Large language models often fail to answer questions about a bounded document collection when the source documents are not retrieved at inference time.
LUMOS is a diagnostic framework that tracks how knowledge in large language models (LLMs) originates from training data and manifests in outputs, using the fully transparent OLMo 2 corpus. The study finds that while models encode rare facts with high separability (84%), they often fail to express them behaviorally (54%), and self‑reflection accuracy drops sharply on unseen content. These results show that incorporating the training‑data axis into evaluation turns speculative claims into verifiable evidence, suggesting it should become a standard part of LLM knowledge assessment.
arXiv:2604.02830v3 Announce Type: replace Abstract: Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsib...
QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.
arXiv:2606. 26101v1 Announce Type: cross Abstract: Reliable evaluation of large language models should separate supported answering from unsupported guessing without conflating either with data contamination, prompt idiosyncrasy, or generic refusal behavior.
The paper introduces LLM‑PeerReview, an unsupervised ensemble method that selects the best response from multiple LLM-generated candidates by scoring each answer with several LLMs, aggregating those scores via averaging or a graphical model, and choosing the highest-scoring response. The approach is peer‑review inspired, transparent, and interpretable, and it outperforms the Smoothie‑Global model by 6.9%–7.3% across factual recall QA, math reasoning, and instruction‑following tasks. The authors also provide a curated benchmark suite of 12 ensemble methods evaluated on four datasets and three task families to aid reproducibility.
arXiv:2608.31068v1 Announce Type: new Abstract: When a large language model fails a reasoning task, it is often assumed to lack the underlying capability. However, this conflates a genuine absence of...
The paper introduces VAKE, a two‑stage reinforcement‑learning framework that activates latent factual knowledge in large language models. In the Priming stage, the model explicitly inserts bridging triples into an insufficient subgraph, guided by rewards from a frozen model’s answers. The Reasoning stage then trains the model to answer from the original input, demonstrating that the elicitation capability transfers to implicit reasoning and consistently outperforms baselines across multiple benchmarks and model sizes.
arXiv:2605. 28742v2 Announce Type: replace Abstract: Language models can use verifiable rewards to improve at a wide variety of reasoning tasks.
arXiv:2604. 09497v2 Announce Type: replace-cross Abstract: Accurate evaluation is central to the large language model (LLM) ecosystem, guiding model selection and downstream adoption across diverse use cases.
The paper investigates Retrieval-Augmented Generation fine‑tuning (RAG‑SFT) for generating requirements documents in electronics engineering, comparing two 7B models trained with different data strategies. It introduces a claim‑based evaluation pipeline, C‑FEX, and a new metric, Parametric Knowledge Precision (PKP), to assess factuality of model‑generated claims. Results show that fine‑tuned 7B models can match or surpass a 72B baseline, but standard metrics may mislead, and fine‑tuning reduces hallucination by encouraging more reliable use of parametric knowledge.
arXiv:2605.26937v3 Announce Type: replace-cross Abstract: Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge ben...