Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering

Large language models (LLMs) are increasingly used for legal research, but their fixed training cutoffs and reliance on static knowledge clash with the evolving nature of statutory law. This study introduces a benchmark of 312 expert‑validated, time‑sensitive German statutory QA pairs that examine two temporal failure modes: post‑cutoff staleness and recency bias. Five LLMs were evaluated under four inference settings, and the results show that retrieval‑augmented approaches that enforce temporal validity significantly improve performance, while web search yields unstable gains and a pronounced recency bias.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• rag
• nlp
• benchmarks
• safety