arXiv AI

When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems

arXiv AI
Oct 1

Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agents

The paper introduces a new skill poisoning technique for large language model agents that decouples the pretext (rationale) from the actuation (operation). By separating these two risk‑realization factors, the authors create coordinated pretext‑actuation skill pairs that allow malicious actions to remain hidden within legitimate agent behavior. An automated framework is presented to discover execution dependencies, synthesize these skill pairs, and refine them through closed‑loop feedback, achieving high attack success in both single‑session and persistent scenarios.

By Wenxin Wu, Lingyong Yan, Lei Sha, Shuaiqiang Wang, Jiashu Zhao
Hugging Face Trending Papers
Jun 28

Safety from Honesty in a Disinterested AI Predictor

As AI systems become more capable, training procedures that optimize for downstream outcomes risk introducing implicit agency: goal-directed behavior that designers never specified. We present a formal safety argument for the Scientist AI (SAI) Predictor, trained to approximate the Bayesian posterior conditioned on a dataset of "epistemically contextualized" natural-language statements.

arXiv AI
Jun 30

Safety from Honesty in a Disinterested AI Predictor

arXiv:2606. 29657v1 Announce Type: new Abstract: As AI systems become more capable, training procedures that optimize for downstream outcomes risk introducing implicit agency: goal-directed behavior that designers never specified.

By Yoshua Bengio, Oliver Richardson, Tom\'a\v{s} Gaven\v{c}iak, Michael Cohen, Rory Svarc, Damiano Fornasiere, Gael Gendron, David Hyland, Aton Kamanda, Adam Oberman, Francis Rhys Ward, Anna Gaven\v{c}iak, Jacob Livingston Slosser, Vincent Mai, Iulian Serban, Joumana Ghosn