arXiv AI
DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 31
DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark
Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence. Although recent studies have exami...
arXiv AI
Aug 17
BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
arXiv:2608. 14161v1 Announce Type: new Abstract: LLMs exhibit social biases that can produce inaccurate and discriminatory inferences, posing risks in high-stakes applications.
arXiv AI
Jun 4
FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
arXiv:2606. 04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks.
arXiv AI
3d ago
Testing the Limits of Truth Directions in LLMs
arXiv:2604.03754v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous...
arXiv Machine Learning
Sep 2
Why Do Reasoning Models Lose Coverage? The Role of Data and Forks in the Road
The paper investigates why fine‑tuned reasoning models lose coverage, observing that pass@k accuracy degrades relative to the base model. The authors attribute this shrinkage to decision‑point or “forks in the road” scenarios in the fine‑tuning data, where the model faces multiple valid reasoning paths. Controlled experiments confirm a strong correlation between such decision‑point prevalence and coverage loss, and show that targeted data synthesis and diversity‑encouraging decoding can partially mitigate the effect.
arXiv AI
Aug 28
Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoning
The paper investigates whether large language models (LLMs) follow Occam's Razor when performing inductive and abductive reasoning. It introduces a synthetic framework for generating questions that require both types of reasoning and a new automated metric to evaluate the simplicity and correctness of generated hypotheses. Experiments show that while LLMs can handle simple scenarios, they struggle with complex world models and producing high‑quality, simplest hypotheses, even when using advanced reasoning techniques.