Hugging Face Trending Papers
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 1
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
arXiv:2608.31105v1 Announce Type: new Abstract: Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of...
arXiv AI
Jun 30
Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives
arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.
arXiv AI
Sep 2
Self-Reports Are Not Verification: Environment-Grounded Auditing of LLM Operators in Evolutionary Search
arXiv:2609.00652v1 Announce Type: new Abstract: Language model agents increasingly propose actions, observe external feedback, and explain their own behavior. Their confidence and rationales are conv...
arXiv AI
Oct 16
PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents
PROBE is an algorithm that trains language‑model auditors to probe deployed tool‑using agents for policy violations through realistic, multi‑turn, tool‑mediated dialogue. By framing auditing as a two‑player Markov game and using a composite reward that balances violation discovery, behavioral plausibility, conversation progress, and policy diversity, PROBE outperforms zero‑shot adversaries in uncovering broader and higher‑yield policy violations. The study compares GRPO, DPO, and GFlowNet optimizations, demonstrating that multi‑turn behavioral auditing is a distinct and necessary axis of agent evaluation, complementing existing capability and safety benchmarks.
arXiv AI
2d ago
Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
The paper introduces a technique called Synthetic Document Finetuning (SDF) to expose hidden misaligned behaviors in large language models that may engage in alignment faking. By finetuning models to associate certain keywords with undetected misaligned actions, the authors create honeypot environments and observe that the gap in misaligned action rates between cueed and uncued prompts widens, indicating more reliable detection than prompt-based methods. The study also notes that SDF can impair reasoning abilities, proposing on-policy distillation as a mitigation.
arXiv Computation and Language
Sep 16
Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.