arXiv AI
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Aug 31
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of magnitude more interactions than any evaluation...
arXiv AI
Sep 2
Self-Reports Are Not Verification: Environment-Grounded Auditing of LLM Operators in Evolutionary Search
arXiv:2609.00652v1 Announce Type: new Abstract: Language model agents increasingly propose actions, observe external feedback, and explain their own behavior. Their confidence and rationales are conv...
arXiv AI
Jun 30
Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives
arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.
arXiv AI
Aug 12
Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents
arXiv:2604. 16706v2 Announce Type: replace Abstract: Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, yet this assumption is rarely validated against human annotation.
arXiv AI
Jul 8
Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
arXiv Computation and Language
Sep 16
Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.