arXiv AI
DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 31
DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark
Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence. Although recent studies have exami...
arXiv AI
Aug 17
BiasTrace: Linking Reasoning Behaviours to Biased Outputs in LLMs
arXiv:2608. 14161v1 Announce Type: new Abstract: LLMs exhibit social biases that can produce inaccurate and discriminatory inferences, posing risks in high-stakes applications.
arXiv AI
Jun 4
FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
arXiv:2606. 04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks.
arXiv Computation and Language
Oct 16
Explanation Generation for Contradiction Reconciliation with LLMs
The paper introduces a new task called reconciliatory explanation generation, where large language models (LLMs) must produce explanations that make contradictory statements compatible. It proposes reusing natural language inference datasets and new quality metrics for scalable evaluation. Experiments with 13 LLMs show limited success, especially when longer reasoning chains are involved, but reinforcement‑learning training improves performance on two narrative reasoning benchmarks.
arXiv AI
Sep 15
Testing the Limits of Truth Directions in LLMs
arXiv:2604.03754v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous...
arXiv Machine Learning
1d ago
Does Reasoning Make Language Models Agree?
The paper investigates how reasoning affects agreement among language models. It distinguishes reasoning as an intervention from reasoning as an observed behavior, comparing agreement under controlled changes in reasoning effort versus observed reasoning lengths. Across nine models and three classification tasks, enabling reasoning increases agreement on the same individuals, while longer reasoning traces correlate with less consistent predictions across models.