← Back to all news
arXiv Statistics ML By Fangran Miao, Shengjie Niu, Jian Huang, Ting Li

Is Your LLM Relay Serving What It Claims? A low-cost and high-power test for LLM Relay Auditing

Read the original on arXiv Statistics ML →

The paper introduces Distributional Fingerprint Auditing (DFA), a statistical method for auditing large language model (LLM) relay services. DFA uses only textual responses from suspect and trusted endpoints, repeatedly querying simple finite-choice probes to build categorical response distributions that serve as fingerprints. By comparing these fingerprints probe‑by‑probe and aggregating evidence with the Cauchy combination test, DFA produces an approximate endpoint‑level p‑value for audit decisions at a chosen significance level, and further supports model attribution and substitution ratio estimation through a likelihood‑based framework.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

Related stories

arXiv Machine Learning
Jul 14

One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions

arXiv:2607. 10252v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly consumed through opaque serving chains - API aggregators, resellers, and inference providers - in which the client has no technical means to confirm that the model answering is the model advertised, and recent audits show that a substantial fraction of commercial endpoints deviate from the vendor's reference weights.

By Tomas Bruckner
llms
More like this →
arXiv AI
Aug 11

Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification

arXiv:2608. 08195v1 Announce Type: cross Abstract: Large language models (LLMs) are high-value assets that can be derived through redeployment, fine-tuning, quantization, or further alignment.

By Yutong Wu, Xiaofan Bai, Shixin Li, Pingyi Hu, Ziqi Zhou, Zilong Wang, Xiaojing Ma, Songfeng Lu, Yuhong Li, Jin Xuan, Yi Wang, Dongmei Zhang, Bin Benjamin Zhu
llmsfine-tuningefficiencysafety
More like this →
arXiv Computation and Language
4d ago

API-TAIL: Detecting Model Substitution in LLM APIs through Temperature-Aligned Likelihood Tails

The paper introduces API-TAIL, a method for detecting partial substitution of large language model APIs. It aligns batch-level temperature effects to standardize surprisal scores, then flags anomalous tail responses that persist under the claimed model. Experiments on commercial APIs show API-TAIL controls false alarms while detecting low-rate substitution more effectively than existing audits.

By Haocheng Mei, Yuchuan Tian, Ye Yuan, Chao Xu, Allen Chen, Kai Han, Yunhe Wang
llms
More like this →