The paper introduces Distributional Fingerprint Auditing (DFA), a statistical method for auditing large language model (LLM) relay services. DFA uses only textual responses from suspect and trusted endpoints, repeatedly querying simple finite-choice probes to build categorical response distributions that serve as fingerprints. By comparing these fingerprints probe‑by‑probe and aggregating evidence with the Cauchy combination test, DFA produces an approximate endpoint‑level p‑value for audit decisions at a chosen significance level, and further supports model attribution and substitution ratio estimation through a likelihood‑based framework.
arXiv:2605. 29524v2 Announce Type: replace-cross Abstract: Relay and reseller APIs increasingly intermediate access to large language models (LLMs), but users have no direct way to verify that a claimed endpoint is actually serving the advertised model.
arXiv:2607. 10252v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly consumed through opaque serving chains - API aggregators, resellers, and inference providers - in which the client has no technical means to confirm that the model answering is the model advertised, and recent audits show that a substantial fraction of commercial endpoints deviate from the vendor's reference weights.
arXiv:2606. 27687v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate, classify, and annotate data whose outputs feed downstream hypothesis tests.
arXiv:2608. 08195v1 Announce Type: cross Abstract: Large language models (LLMs) are high-value assets that can be derived through redeployment, fine-tuning, quantization, or further alignment.
The paper introduces API-TAIL, a method for detecting partial substitution of large language model APIs. It aligns batch-level temperature effects to standardize surprisal scores, then flags anomalous tail responses that persist under the claimed model. Experiments on commercial APIs show API-TAIL controls false alarms while detecting low-rate substitution more effectively than existing audits.
arXiv:2609.00052v1 Announce Type: cross Abstract: Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to...
arXiv:2606. 16100v1 Announce Type: cross Abstract: As Large Language Model (LLM) APIs become ubiquitous, users increasingly rely on black-box fingerprinting to verify that providers are serving the advertised premium models.
arXiv:2603. 23171v3 Announce Type: replace-cross Abstract: Providers monitor deployed large language models (LLMs) to detect misuse that they cannot prevent.
arXiv:2606. 03330v1 Announce Type: cross Abstract: Literature reveals that a Large Language Model's (LLM) behavior is not only conditioned by its original weights but also its instance-level parameters, such as instructional prompt, sampling configuration or quantization.
arXiv:2607. 18826v1 Announce Type: cross Abstract: LLM-agent defenses are typically evaluated one session at a time.
The paper introduces CTF-ABACUS, a trace-based auditing framework that reconstructs each autonomous language-model agent’s run in Capture-the-Flag (CTF) challenges into evidence‑grounded solve profiles. By decomposing actions into penetration‑testing phases and techniques, it distinguishes genuine exploitation from shortcut methods such as memorized recall or guessing. Applying the framework to 1,435 CTF attempts by six models on 240 challenges shows that only 62‑87% of recovered flags are trace‑verified, highlighting that many successes rely on shallow trajectories rather than true exploitation.
arXiv:2608. 05199v1 Announce Type: cross Abstract: Autonomous security agents operate as staged pipelines, such as classifying network traffic and then attributing attacks to a specific technique.