arXiv AI

Auditing Proprietary Alignment in Large Language Models: A Comparative Framework Without a Ground-Truth Standard

arXiv:2606. 08381v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly released and deployed through opaque development and deployment pipelines, enabling model providers to inject intentional, provider-specific policies without officially announcing them.

arXiv Computation and Language
Sep 21

Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guarantees

Conformal Privacy Auditing (CPA) is a distribution‑free framework that calibrates re‑identification risk for each released document against large language model (LLM)‑empowered adversaries. It outputs a conformal ambiguity set of candidate identities that is guaranteed to contain the true identity with a user‑chosen confidence level under exchangeability, along with an interpretable leakage proxy derived from the set size. CPA supports both logit‑access and sampling‑only attackers, enabling audits of both open‑source and proprietary models, and demonstrates calibrated coverage across various benchmarks and attacker configurations.

By Shuo Huang, Gholamreza Haffari, Xingliang Yuan, Ting Yu, Lizhen Qu
arXiv Machine Learning
Sep 11

Black-Box Membership Inference via Word-Level Probability Estimation

The paper introduces Word-level Probability MIA (WPMIA), a black-box membership inference attack that estimates word-level generation probabilities via Monte Carlo sampling and local kernel smoothing, then aggregates them into a sequence-level likelihood estimator. By conditioning on different prefixes, WPMIA amplifies distributional differences between member and non-member texts, outperforming existing black-box baselines on open-source LLMs and achieving an average TPR@5%FPR of 42.0 on proprietary models such as GPT‑5‑Chat, Gemini‑2.5‑Flash, and Claude‑4.5‑Haiku.

By Shengjie Niu, Yeheng Ge, Jian Huang
arXiv Machine Learning
Jul 14

One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions

arXiv:2607. 10252v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly consumed through opaque serving chains - API aggregators, resellers, and inference providers - in which the client has no technical means to confirm that the model answering is the model advertised, and recent audits show that a substantial fraction of commercial endpoints deviate from the vendor's reference weights.

By Tomas Bruckner
arXiv AI
6d ago

A Survey on Fake Review Detection: From Pre-trained Language Models to Large Language Models

The article surveys fake review detection research, focusing on how pre‑trained language models (PLMs) and large language models (LLMs) influence both the generation of deceptive reviews and their detection. It reviews 211 studies from 2018 to early 2026, categorizing methods by evidence source—such as review text, sentiment, rating behavior, temporal metadata, user‑product graphs, multimodal content, external knowledge, and LLM‑generated signals—and by fusion level. The survey traces the evolution from traditional machine learning to PLM‑based and LLM‑based approaches, evaluates performance on Amazon, Yelp, and OpSpam benchmarks, and highlights open challenges including adversarial generation, cross‑domain transfer, uncertainty‑aware fusion, robustness to missing sources, interpretability, and trustworthy evaluation of AI‑generated deceptive content.

By Fanji Yang (Guizhou University of Finance and Economics), Huiyao Chen (Harbin Institute of Technology), Xi Yu (Guizhou University of Finance and Economics), Meishan Zhang (Harbin Institute of Technology), Xiaohong Xiao (Guizhou University of Commerce), Mingsen Deng (Guizhou University of Finance and Economics)