arXiv AI
Aug 5
Risky Business: Measuring The Faithfulness-Safety Tension
arXiv:2608. 03745v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning offers a promising window into model monitoring.
Do Thinking Tokens Help with Safety?
arXiv:2606. 25013v1 Announce Type: new Abstract: Today's reasoning models use thinking tokens to attain stronger performance on benchmarks than their instruction-tuned counterparts.
Related stories
arXiv AI
Aug 18
PL-Guard: Probabilistic Logic Reasoning for LLM Guardrails
arXiv:2608. 15673v1 Announce Type: cross Abstract: Large language model guardrails can be viewed as policy-consistency problems: a system must determine which policy-relevant facts hold in a prompt-response pair and what those facts imply under a given policy.
arXiv AI
Sep 17
First Token Matters: Understanding Safety Collapse in Large Reasoning Models
The paper investigates why large reasoning models (LRMs) lose safety alignment when faced with harmful queries. By analyzing token-level refusal dynamics, the authors identify a vulnerability called Onset Refusal Collapse (ORC), where the refusal signal drops sharply at the first generated token, leading to unsafe responses. They introduce SafeToken, a lightweight inference-time intervention that injects a learned safety anchor at reasoning onset, which mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility.
Hugging Face Trending Papers
5d ago
EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
EOPSA (Efficient On-Policy Self-Distilled Safety Alignment) addresses inefficiencies in On-Policy Self-Distillation (OPSD) for safety alignment by focusing training on safety-critical tokens. It introduces Adaptive Rollout Scheduling, which limits generation length based on a Teacher Rescue Rate metric, and Selective Distillation, which filters out safety-neutral tokens to concentrate gradient updates on safety-pivotal transitions. Experiments on models up to 32B parameters show that EOPSA reduces rollout computation by about 50% and backpropagates through only roughly 2% of tokens, outperforming full-token distillation baselines in safety compliance and reasoning retention.
arXiv AI
2d ago
When Reasoning Helps Action: Monitoring and Steering Chain-of-Thought in Vision-Language-Action Policies
arXiv:2610.00601v1 Announce Type: cross Abstract: Reasoning-enabled VLA policies expose chain-of-thought (CoT) traces that appear to explain and guide their actions, creating a potential interface fo...
arXiv AI
Jun 30
Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
arXiv:2606. 30128v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditioning on more tokens buys extra computation before the model commits to an answer?
arXiv AI
Jun 3
Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
arXiv:2606. 02835v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) improve performance by generating explicit intermediate reasoning traces through increased test-time compute, yet the assumption that longer reasoning is consistently beneficial remains under-examined.
arXiv AI
Jun 11
Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
arXiv:2606. 11211v1 Announce Type: cross Abstract: The ability of large language models (LLMs) to express calibrated uncertainty is important for safe deployment.
arXiv AI
Jul 24
Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers
arXiv:2603. 01437v2 Announce Type: replace Abstract: As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning.
arXiv AI
Sep 4
</think> Doesn't Stop Reasoning: Analysis of Spurious CoT Termination
The paper investigates a training‑free early‑exit technique that inserts an end‑of‑think (EoT) token to terminate chain‑of‑thought (CoT) reasoning in large reasoning models. It finds that the injected EoT often fails to cleanly switch the model from reasoning to answering, leading to continued reasoning‑like generation—termed spurious CoT termination—whose length scales with the amount of reasoning saved. By increasing attention to the EoT token through Exit‑token Attention Biasing (EAB), the authors reduce spurious termination and shorten the answering phase across multiple models and benchmarks.
arXiv AI
Aug 3
On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
arXiv:2607. 29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought.
arXiv AI
Jul 14
Valid $\ne$ Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought
arXiv:2607. 11266v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Large Language Models (LLMs), yet it often incurs substantial computational costs due to over-reasoning: the generation of redundant, verbose, or irrelevant steps.