arXiv:2608.30413v1 Announce Type: new Abstract: Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of n...
arXiv:2608. 14161v1 Announce Type: new Abstract: LLMs exhibit social biases that can produce inaccurate and discriminatory inferences, posing risks in high-stakes applications.
arXiv:2606. 04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks.
The paper introduces a new task called reconciliatory explanation generation, where large language models (LLMs) must produce explanations that make contradictory statements compatible. It proposes reusing natural language inference datasets and new quality metrics for scalable evaluation. Experiments with 13 LLMs show limited success, especially when longer reasoning chains are involved, but reinforcement‑learning training improves performance on two narrative reasoning benchmarks.
The paper introduces VERITAS, a method that verbalises a language model’s internal computation during chain‑of‑thought (CoT) generation by projecting layer‑wise Jacobians to produce a token‑by‑token account of reasoning evolution. By training a classifier on these verbalised features, VERITAS detects where and how reasoning diverges toward unfaithfulness, achieving a 28.4% balanced‑accuracy gain over LLM‑as‑Judge and a 16.0% improvement over activation probing across four datasets. The approach demonstrates generalisability to different datasets and model architectures, offering a richer, more interpretable lens on CoT faithfulness that moves beyond detection toward mechanistic insight.
The paper investigates how reasoning affects agreement among language models. It distinguishes reasoning as an intervention from reasoning as an observed behavior, comparing agreement under controlled changes in reasoning effort versus observed reasoning lengths. Across nine models and three classification tasks, enabling reasoning increases agreement on the same individuals, while longer reasoning traces correlate with less consistent predictions across models.
The paper investigates why fine‑tuned reasoning models lose coverage, observing that pass@k accuracy degrades relative to the base model. The authors attribute this shrinkage to decision‑point or “forks in the road” scenarios in the fine‑tuning data, where the model faces multiple valid reasoning paths. Controlled experiments confirm a strong correlation between such decision‑point prevalence and coverage loss, and show that targeted data synthesis and diversity‑encouraging decoding can partially mitigate the effect.
The paper introduces a method for analyzing how interventions in a prompt affect a language model’s reasoning by using behavioral counterfactuals. It trains a module to edit the residual stream so that the next-token distributions for a fixed response match between a source and a counterfactual prompt, outperforming standard activation reconstruction. The resulting attribution maps and visualizations help identify causally important locations for counterfactual recovery and guide hypothesis testing with activation patching, as demonstrated in a case study on Gemma-4-E4B-it and DeepSeek-R1-0528-Qwen3-8B.
arXiv:2601. 22642v2 Announce Type: replace Abstract: Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic systems avoid.
Faithfully understanding the internal process that occurs as models form their chain of thought is an important problem with the rise in model capabilities. However, this remains challenging due to th...
GRACE is a step‑level benchmark for evaluating the faithfulness of chain‑of‑thought reasoning over context. It provides human annotations for each step in CoT traces from 10 models across 4 datasets, labeling faithfulness, error category, and natural‑language explanations. The benchmark introduces a data‑driven taxonomy that splits errors into GRACE‑Inference (deductive) and GRACE‑Grounding (factual) tracks, each with four categories, and demonstrates that incorporating step‑level faithfulness signals can improve downstream accuracy and reasoning reliability.
arXiv:2604.27251v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) acquire reasoning capabilities through shared inference patterns in pre-training data, which are further elicite...