arXiv AI
Chain of Thought Monitoring Can Fail Even When CoT Is Necessary
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 24
Diagnosing Pathological Chain-of-Thought in Reasoning Models
arXiv:2602. 13904v2 Announce Type: replace Abstract: Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety.
arXiv Machine Learning
3d ago
Monitorability Is Trainable: Post-Training for Chain-of-Thought Monitorability
The paper introduces an unsupervised post‑training objective that maximizes mutual information between chain‑of‑thought (CoT) explanations and model outputs to enhance CoT monitorability. Experiments on three model organisms show that this method recovers most or all lost monitorability in degraded CoT settings and further improves monitorability, unsafe behavior prediction, and consequential reasoning in undegraded policies. The results argue that monitorability should be treated as a trainable attribute and that a dedicated post‑training stage can protect CoT from direct optimization pressure.
arXiv AI
6d ago
OwlSight: A Panoramic Monitor for Chain-of-Thought Reasoning
OwlSight is a new chain‑of‑thought (CoT) monitoring system that uses a customizable Factor strategy to adapt to a wide range of tasks and monitoring needs. By incorporating fine‑grained and coarse‑grained factors, OwlSight can detect specific behaviors such as email leakage or broader privacy concerns across multiple dimensions like capability, safety, and trustworthiness. Experiments on 15,971 reasoning trajectories show that OwlSight outperforms general‑purpose LLMs by 1.32‑30.57 points at fine granularity and 11.71‑27.81 points at coarse granularity, achieving state‑of‑the‑art results on OpenAI’s CoT monitorability benchmark and demonstrating robustness to model changes, factor rephrasings, CoT compression, and obfuscation.
arXiv AI
Aug 11
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
arXiv:2603. 28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs.
arXiv AI
Jul 9
Reasoning Consistency Scanning: A Framework for Auditing Chain-of-Thought Validity in AI Safety Evaluations
arXiv:2607. 07229v1 Announce Type: new Abstract: Prior work has shown that chain-of-thought (CoT) reasoning is often unfaithful: a model's stated reasoning does not reliably reflect the process that produced its output.
arXiv Machine Learning
Aug 5
Evaluation Blindness: How Silent Measurement Failures Corrupt AI Systems from Training to Deployment
arXiv:2608. 02786v1 Announce Type: new Abstract: AI systems can fail silently.