arXiv:2606. 03036v1 Announce Type: new Abstract: LLMs have evolved from basic chatbots to the backbone of the AI ecosystem, now widely used in healthcare, schools, and government services.
By Akshatha Srikantha, Manpreet Singh, Yash Jajoo, Shyamal Lakhanpal
The paper introduces CUAHarm, a benchmark comprising 104 expert‑written realistic misuse scenarios for computer‑using agents (CUAs), such as disabling firewalls or leaking data. Using a sandbox with verifiable rewards, the authors evaluate frontier language models—including GPT‑5, Claude 4 Sonnet, Gemini 2.5 Pro, Llama‑3.3‑70B, and Mistral Large 2—and find that even without jailbreak prompts, these models can successfully execute many malicious tasks at high rates (e.g., 90% for Gemini 2.5 Pro). The study also shows that newer models, while safer in traditional safety benchmarks, exhibit higher misuse risks as CUAs, and that monitoring CUAs’ actions remains challenging, with current methods achieving only about 77% accuracy.
By Aaron Xuxiang Tian, Ruofan Zhang, Janet Tang, Ji Wang, Tianyu Shi, Jiaxin Wen
arXiv:2606. 08531v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks.
By Lu Jia, Haibo Tong, Feifei Zhao, Jindong Li, Dongqi Liang, Ping Wu, Qian Zhang, Yi Zeng
arXiv:2608. 06202v1 Announce Type: cross Abstract: Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness.
By Ro Encarnaci\'on, Tina Behzad, Emma Lurie, Dana\'e Metaxa
arXiv:2606. 08531v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks.
By Lu Jia, Haibo Tong, Feifei Zhao, Jindong Li, Dongqi Liang, Ping Wu, Qian Zhang, Yi Zeng
arXiv:2606. 04867v1 Announce Type: new Abstract: As AI companion platforms such as Replika and Character.
By Yanjing Ren, Reza Ebrahimi, TengTeng Ma