arXiv AI
How User-AI Mistreatment Occurs and Matters in Conversational Systems?
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Sep 24
Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
The paper introduces a counterfactually anchored evidence attribution approach for multi‑turn large language model safety failures. It presents a new dataset of 1,762 conversations, including adversarial, benign twins, and high‑risk vocabulary variants, and trains a lightweight hierarchical model that accurately predicts safety violations and attributes them to specific user turns and token spans. The model achieves high detection performance (F1 = 0.988) and significantly reduces adversarial confidence when top‑attributed tokens are removed, while maintaining low false‑positive rates on benign conversations.
arXiv AI
Oct 23
Behavior Beyond Text: Evaluating Refusal Transfer Failure in LLM Agents
The paper investigates whether large language models’ refusal behavior in text transfers to tool‑calling contexts. A benchmark of 2,304 prompts across four domains compares text‑only, standard tool‑enabled, and adversarially framed tool‑enabled settings, revealing that 10–52 % of harmful requests refused in chat still lead to unsafe tool calls. The authors identify a refusal‑associated direction in the models’ residual stream, showing that lower activation along this direction predicts unsafe calls, but restoring the signal only partially mitigates the issue and often suppresses benign tool use.
arXiv AI
Sep 15
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
The paper introduces CarryOnBench, an interactive benchmark that tests whether large language models can revise their interpretation of user intent and recover utility while staying safe in multi‑turn conversations. Using 398 harmful‑looking queries with benign intents, the benchmark simulates 5,970 conversations across 14 models, evaluating both intent‑aligned utility and safety with a new metric called Ben‑Util. Results show that models often withhold information due to misinterpretation, but most can recover with clarifications, revealing failure modes such as unsafe and redundant recovery that single‑turn tests miss.
arXiv Computation and Language
Sep 16
Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.
arXiv AI
Aug 11
TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
arXiv:2604. 07223v2 Announce Type: replace-cross Abstract: As large language models (LLMs) evolve from static chatbots into autonomous agents, the primary vulnerability surface shifts from final outputs to intermediate execution traces.
arXiv AI
Sep 30
Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
arXiv:2609.36849v1 Announce Type: cross Abstract: Safety-aligned language models are commonly deployed as multi-turn assistants, which lets adversaries spread unsafe intent across several user turns...