arXiv AI By Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li, Jianbing Ni

What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays

Read the original on arXiv AI →

The paper evaluates 20 third‑party API relays that provide access to large language models such as GPT‑6 Astra and Claude Opus 5. It finds that many low‑price relays exhibit answer‑behavior differences, some return tool calls even when disabled, and several report inflated input‑token counts leading to higher charges than the official API. The study demonstrates that advertised models and prices alone do not guarantee reliable service, highlighting the need for repeated, detailed evaluations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Nov 5

What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays

The paper evaluates 20 third‑party LLM API relays that claim to provide GPT‑6 Astra and Claude Opus 5 at lower prices. By comparing answers to official responses, inspecting tool permissions, and reconciling billed token counts, the study finds that many relays exhibit behavioral differences, return unexpected tool calls, and report inflated input‑token usage. These discrepancies highlight that advertised models and prices alone do not guarantee reliable service.

arXiv Computation and Language
Oct 28

API-TAIL: Detecting Model Substitution in LLM APIs through Temperature-Aligned Likelihood Tails

The paper introduces API-TAIL, a method for detecting partial substitution of large language model APIs. It aligns batch-level temperature effects to standardize surprisal scores, then flags anomalous tail responses that persist under the claimed model. Experiments on commercial APIs show API-TAIL controls false alarms while detecting low-rate substitution more effectively than existing audits.

By Haocheng Mei, Yuchuan Tian, Ye Yuan, Chao Xu, Allen Chen, Kai Han, Yunhe Wang
arXiv Statistics ML
Oct 28

Is Your LLM Relay Serving What It Claims? A low-cost and high-power test for LLM Relay Auditing

The paper introduces Distributional Fingerprint Auditing (DFA), a statistical method for auditing large language model (LLM) relay services. DFA uses only textual responses from suspect and trusted endpoints, repeatedly querying simple finite-choice probes to build categorical response distributions that serve as fingerprints. By comparing these fingerprints probe‑by‑probe and aggregating evidence with the Cauchy combination test, DFA produces an approximate endpoint‑level p‑value for audit decisions at a chosen significance level, and further supports model attribution and substitution ratio estimation through a likelihood‑based framework.

By Fangran Miao, Shengjie Niu, Jian Huang, Ting Li
arXiv Machine Learning
Jul 14

One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions

arXiv:2607. 10252v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly consumed through opaque serving chains - API aggregators, resellers, and inference providers - in which the client has no technical means to confirm that the model answering is the model advertised, and recent audits show that a substantial fraction of commercial endpoints deviate from the vendor's reference weights.

By Tomas Bruckner
arXiv Computation and Language
Sep 1

Token Counts Are Not Model Lineage: A Frozen-Threshold Holdout Study of Black-Box LLM API Fingerprinting

The study evaluates whether prompt‑token counts can reliably identify the lineage of large language models served via APIs. Using a frozen‑threshold approach on 24 labeled endpoint pairs, the authors find that token‑count consistency perfectly separates development pairs but only half of the holdout pairs meet the strict repeatability criteria, yielding moderate accuracy and perfect specificity. The results confirm token‑count consistency as a fingerprint of shared tokenization stacks but reject it as a standalone test for model‑family attribution.

By Bo Chen