arXiv AI September 1, 2026 By Jason Luo, Saibilila Abudukelimu, Judy Song, Andrew Feng, Shivank Garg, Vasu Sharma, Kevin Zhu

MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects

Read the original on arXiv AI →

MUDDLE is a benchmark designed to disentangle the effects of document length and topical distractors on document question‑answering systems. It contains 270 human‑annotated questions, each tested in five conditions: the source alone, the source with two or four hard negatives (topically similar), and the source with two or four random distractors matched in length and provenance. Experiments with GPT‑5‑mini show that hard negatives reduce accuracy more than length‑matched random distractors, indicating that topical similarity is a more significant source of error than length alone.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 21

The First Drop of Ink: Nonlinear Impact of Distracting Information in Long-Context Reasoning

arXiv:2605. 10828v2 Announce Type: replace Abstract: As large language models are increasingly deployed in retrieval-augmented generation and agentic systems that accumulate extensive context, understanding how distracting information affects long-context performance becomes critical.

By Muhan Gao, Zih-Ching Chen, Kuan-Hao Huang

llms rag agents

arXiv AI

Aug 5

Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks

arXiv:2608. 03297v1 Announce Type: new Abstract: A standard claim in the literature on retrieval-augmented and memory-augmented language models is that shorter context is better when the relevant information is preserved.

By Mohsen Arjmandi

llms rag benchmarks

arXiv Machine Learning

Sep 22

Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models

arXiv:2609.22101v1 Announce Type: cross Abstract: Large language models can process increasingly long prompts, yet their ability to locate and use decisive evidence may degrade as irrelevant or confu...

By Meysam Ghaffari, Nina Fatehi, Bhaskar Sen, Nasim Sabetpour, Carlos Morato

llms

Hugging Face Trending Papers

Oct 29

Elucidating the Design Space of LM Data Attribution

The paper introduces the Controlled Attribution Benchmark (CATT), a synthetic evaluation framework that fine‑tunes language models on entirely fictitious knowledge to provide clear ground truth for data attribution. By systematically varying single dataset properties—such as document redundancy and lexical distractors—CATT reveals that these factors consistently degrade attribution performance. The study also explores two underexamined design choices: using negative log‑odds gradients instead of cross‑entropy gradients, and aggregating only the top token attribution scores, which together improve ground‑truth recall by 35–72 percentage points and enhance counterfactual retraining performance on FTRACE‑TREX by up to 4.9×.

llms fine-tuning benchmarks

arXiv Machine Learning

Sep 30

Sieve and Sage: Efficient Distraction Filtering for Reliable RALM Abstention

arXiv:2609.35794v1 Announce Type: cross Abstract: Just as Socrates recognized the limits of his own knowledge, Retrieval-Augmented Language Models (RALMs) should learn to abstain when the retrieved e...

By Jongbin Won, Sung Geun An, Jay-yoon Lee

llms rag safety

arXiv Computation and Language

Oct 21

The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

arXiv:2610.26951v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunk...

By Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum, Zhuohan Xie, Preslav Nakov, Veselin Stoyanov

llms rag nlp benchmarks