arXiv AI
Interpreting Reasoning of Large Language Models via Partial Information Decomposition
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
arXiv:2508. 02178v3 Announce Type: replace Abstract: Large reasoning models (LRMs) often exhibit overthinking, producing verbose Chain-of-Thought (CoT) traces that increase inference cost and obscure the underlying reasoning process.
arXiv AI
Jul 14
Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought
arXiv:2607. 11266v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps.
arXiv AI
Jun 8
DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling
arXiv:2606. 07108v1 Announce Type: new Abstract: Recent advances in Large Reasoning Models (LRMs) demonstrate remarkable performance improvements by iteratively reflecting, exploring, and executing complex tasks, yet suffer from inefficiencies due to redundant reasoning, known as "overthinking".
arXiv AI
Sep 17
OBC-Prune: Outcome-Based Calibration for Large Reasoning Model Pruning
OBC‑Prune introduces an outcome‑based calibration approach for pruning large reasoning models, focusing on the causal importance of each reasoning sentence rather than uniform activation salience. By pairing correct and incorrect rollouts and using intervention‑based analysis, it assigns per‑token weights that guide one‑shot pruning methods such as SparseGPT, Wanda, and ALPS. Experiments on DeepSeek‑R1‑Distill‑Qwen models show consistent accuracy gains and shorter reasoning traces across multiple benchmarks at 40‑50% sparsity.
arXiv Machine Learning
Aug 11
Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond
arXiv:2510. 19990v2 Announce Type: replace Abstract: The reasoning paradigm, where language models reason before answering, has enabled breakthroughs on tasks such as mathematical problem-solving.
arXiv Machine Learning
Sep 24
Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning
The paper introduces RECAP, a redundancy-aware credit assignment method that improves reasoning efficiency in large language models by assigning credit to each reasoning step based on its downstream role and contribution to the correct answer. RECAP uses a semantic dependency graph to measure structural responsibility and evaluates step efficacy via changes in gold-answer log-likelihood, enabling step-specific updates without requiring a separate reward model or concise trajectories. Experiments on two 7B models across four mathematical reasoning benchmarks show that RECAP enhances the accuracy-efficiency trade-off, boosting pass@1 by 2.0–3.7 percentage points while cutting reasoning tokens by 8–31% compared to GRPO.