The paper introduces CauTrace, a white‑box diagnostic framework that analyzes why large language models (LLMs) fail by examining internal states, representation propagation, and temporal coordination during reasoning. It defines seven functional states to capture support and constraint processing, tracks how these states evolve across Transformer blocks, and studies the timing of commitment and verification steps. Experiments on Qwen and LLaMA-family models show distinct failure signatures, and repair/steering tests confirm that the framework’s signals align with expected changes in the inference process.
arXiv:2606. 07808v1 Announce Type: new Abstract: Reasoning language models deployed in agentic workflows must follow an instruction hierarchy: when instructions from different sources conflict, the model should obey the highest-privilege applicable instruction.
arXiv:2607. 07903v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit remarkable capabilities but remain highly vulnerable to adversarial prompts and jailbreak attacks.
arXiv:2606. 10740v1 Announce Type: new Abstract: Failures in multi-turn reasoning models are largely invisible to terminal-score evaluation.
arXiv:2606. 06635v1 Announce Type: cross Abstract: Failures in language model reasoning emerge through distinct processes that leave identifiable signatures in the reasoning trace.
arXiv:2608. 03291v1 Announce Type: cross Abstract: Chain-of-thought (CoT) reasoning improves large language model (LLM) performance while also providing an observable interface to the model's reasoning process.
arXiv:2606. 22686v2 Announce Type: replace-cross Abstract: Modern Large Language Models (LLMs) rely on extensive safety alignment, yet the mechanistic basis of refusal remains opaque.
arXiv:2605.17467v2 Announce Type: replace Abstract: Large language model-driven multi-agent systems (LLM-MAS) excel at complex tasks, yet unreliable agents remain a key bottleneck to system-level rel...
arXiv:2606. 24589v1 Announce Type: new Abstract: Scaling adversarial evaluation of large language models requires both a method for generating hard inputs and a reliable way to confirm that resulting failures are real.
arXiv:2608. 03745v1 Announce Type: new Abstract: Chain-of-Thought (CoT) reasoning offers a promising window into model monitoring.
The paper investigates automated failure attribution (AFA) in large language model (LLM)-based agentic systems, where multiple agents collaborate on complex tasks. It finds that LLMs contain sufficient internal knowledge to attribute failures but often fail to output the correct step, revealing a knows‑but‑won’t‑say gap. To address this, the authors introduce grafting—a manual intervention that leverages latent model knowledge—showing it consistently outperforms direct prompting across several LLMs and benchmarks.
The paper investigates how safety‑aligned large language models (LLMs) can still be tricked by jailbreaks or over‑refuse benign requests. By analyzing residual‑stream updates in decoder‑only Transformers, the authors find that harmfulness is more readily decoded from attention updates—especially in early layers—while the MLP component is primarily responsible for writing refusal responses. Post‑training enhances MLP‑driven refusal writing without improving harmfulness decodability, and causal interventions confirm that downstream MLP updates are crucial for the refusal shift caused by upstream attention changes. Building on these insights, the authors propose Sentinel, a system that combines an attention‑derived perception score with an MLP‑derived decision score to better detect jailbreaks and over‑refusals across multiple models.
arXiv:2607. 18366v1 Announce Type: new Abstract: Large language models (LLMs) serving as planners in tool-using autonomous agents introduce dynamic reliability risks in multi-turn execution.