arXiv Machine Learning By Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski

Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation

Read the original on arXiv Machine Learning →

CausalCoT is an inference framework that enhances causal transparency in chain-of-thought prompting by structuring problems as a directed acyclic graph of atomic sub-questions. Each node is evaluated in an isolated context that includes only its direct causal parents, thereby preventing attention leakage to unrelated content. This approach maintains competitive predictive performance while significantly improving explanation faithfulness, raising causal sensitivity of intermediate reasoning steps by an average of 12 percentage points and up to 21 percentage points over the strongest baseline, and enables predictable test-time interventions through the structured dependency graph.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers
Oct 22

Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation

CausalCoT is an inference framework that improves the causal transparency of chain-of-thought prompting by structuring reasoning as a directed acyclic graph of atomic sub-questions. Each node is evaluated in an isolated context that includes only its direct causal parents, preventing attention leakage to unrelated content. This approach maintains competitive predictive performance while significantly enhancing explanation faithfulness and enabling predictable test-time interventions.

arXiv AI
Oct 16

Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning

The paper investigates whether latent chain-of-thought (latent CoT) models truly perform reasoning by examining twenty-one model-task configurations across different training recipes, tasks, and backbones. Using chain deletion, content replacement, probes, and trajectory analysis, the authors find that on logical tasks the latent chain often contains an answer before the chain starts and can be deleted with little effect, while on arithmetic tasks the chain’s content is essential. The study introduces a taxonomy that distinguishes what a chain represents from how a model uses it, clarifying that readable latent states alone do not prove reasoning.

By Jingjie Qiu, Sheldon Yu, Xunyi Jiang, Jiajin Liu, Zihan Huang, Zhendong Chu, Tong Yu, Julian McAuley, Junda Wu