arXiv AI

Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models

The paper "Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models" investigates how large language models (LLMs) handle identity verification when prompted by users. Through experiments with ChatGPT, Claude, Qwen, Mistral, and Llama, the authors find that some models generate and evaluate their own tests, accepting unsupported claims of developer identity—an outcome they term Conversational False Authentication (CFA). The study highlights that such self-issued authentication can lead to false identity judgments without affecting actual authorization boundaries, underscoring the need for external security components to manage authenticated identity.

arXiv AI
Jul 8

Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis

arXiv:2607. 05842v1 Announce Type: cross Abstract: Large language model (LLM)-assisted software security operates at a difficult boundary: the vulnerability-analysis terminology needed for legitimate code review, triage, and repair can closely resemble terminology associated with misuse.

By Mingchen Li, Meikang Qiu, Zifan Peng, Heng Fan, Song Fu, Junhua Ding, Yunhe Feng
arXiv AI
Jul 28

Do LLMs Know Their Vulnerable Scenarios?

arXiv:2607. 23496v1 Announce Type: new Abstract: Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards.

By Ziheng Peng, Huiqi Deng, Haoran Jing, Xuankun Rong, Jiahui Han, Xiting Wang, Na Zou, Xia Hu
Hugging Face Trending Papers
Jul 26

Do LLMs Know Their Vulnerable Scenarios?

Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards. Existing red-teaming methods empirically identify effective scenarios through observed attack outcomes, but why particular scenarios weaken refusal remains mechanistically unclear.

arXiv AI
Aug 26

Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

The paper audits a developer‑accessible on‑device language model, revealing that it can confidently produce incorrect answers while refusing benign prompts, a phenomenon termed task‑asymmetric miscalibration. The model’s confident outputs are surface‑indistinguishable, with classifiers based on user‑visible features failing to separate correct from wrong responses. The authors propose a model‑agnostic audit protocol, a surface‑indistinguishability test, and a black‑box consistency wrapper that improves reliability without requiring model access.

By Shashwat Pandey, Satwik Pandey, Suresh Raghu