arXiv:2608. 04591v1 Announce Type: cross Abstract: Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context.
arXiv:2610.37162v1 Announce Type: cross Abstract: Coverage-evaluation benchmarks quantify whether retrieved evidence covers the different pieces of information needed to satisfy an information need....
Long-context evaluations often test whether a model can recover distant evidence, but recoverability does not guarantee behavioral influence. We test the prediction that a distant discourse anchor can...
arXiv:2606. 27383v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as research assistants, yet it remains unclear whether they can calibrate research takeaways to the strength and scope of the supporting evidence.
arXiv:2609.22939v1 Announce Type: cross Abstract: Long-context models read a novel the way a person reads a printout: one token after another, in narrative order, with the whole history competing for...
The paper introduces MWE‑ECL, a bilingual diagnostic framework that tests whether distant discourse anchors can override local lexical priors in multi‑word expression interpretation. It evaluates models on a 0‑128K context grid, finding that while retrieval of anchors is near perfect, the ability to change locally preferred readings varies, especially when the model’s default conflicts with the anchor. The study shows that explicit recoverability does not always translate into behavioral influence, with gaps differing across models and languages.
arXiv:2610.26951v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...
arXiv:2604. 04593v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions.
PROOF is a benchmark that profiles the reliability of object-level facts in instruction-tuned language models by converting a frozen Wikidata snapshot into 18,486 English multiple-choice questions grounded in 11,779 semantic facts across 101 classes, 392 properties, and 14 domains. Each question includes an explicit "I don't know" option, a "No correct option" control, and nine controlled formulations, with 1,849 questions designed as no-correct-option traps. The study evaluates 18 open-weight model deployments on 166,374 prompts, revealing wide variability in factual accuracy, sensitivity to wording changes, and the impact of decoder perturbations.
arXiv:2609.37469v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external sources, but retrieved passages often name the right entities without...
The study examines how different editorial framings in prompts influence large language models’ statistical analysis reports. Using a 4×4 factorial design, researchers found that certain framings—particularly brutally critical prompts on genuine effects and significance-seeking prompts on underpowered nulls—led to factual misrepresentations. Tone shifts were more widespread, with critical framing inducing defensive language across all data patterns, while a confound in the data largely prevented both factual and tonal distortions.
The paper introduces LURCH, a benchmark of 6,550 question‑answering instances that quantifies long‑context reasoning difficulty using three properties: evidence scope, evidence dispersion, and reasoning depth. Experiments with both proprietary and open‑weight models show that these properties are strongly negatively correlated with performance, that performance drops sharply for dispersed evidence as context length grows, and that difficulty increases when multiple properties are high simultaneously. The study demonstrates that context length alone is insufficient to explain model failures, and that the new metrics provide a more nuanced understanding of long‑context challenges.
arXiv:2610.07894v1 Announce Type: cross Abstract: Large language models (LLMs) are expected to answer questions faithfully based on the provided context, abstaining when the context information is in...