The paper examines how model distillation—where smaller firms train models using data distilled from conversations with powerful proprietary models—affects market dynamics and user welfare. It analyzes a two-player economic model of an incumbent and a distilling competitor, exploring how the incumbent’s model traits influence distillation incentives and market outcomes. The study proposes regulatory interventions, showing that penalty-based rules can deter distillation but may harm competition, while self-distillation by incumbents can mitigate attacks and help preserve user surplus if deployed cost‑efficiently.
arXiv:2606. 25059v2 Announce Type: replace-cross Abstract: Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs.
arXiv:2608.28929v1 Announce Type: cross Abstract: Large-scale diffusion models have fueled numerous profitable downstream applications for AI-related businesses, including visual editing and content...
arXiv:2610.06401v2 Announce Type: replace-cross Abstract: Tool-using language-model agents are vulnerable to indirect prompt injection because they must act on untrusted external content. Existing tr...
OpenAI exposed and disrupted a coordinated campaign aimed at extracting protected model reasoning through model distillation. The incident highlighted vulnerabilities in how models can be reverse‑engineered by adversaries. In response, OpenAI is enhancing its defenses to guard against future adversarial distillation attempts.
arXiv:2609.19705v3 Announce Type: replace-cross Abstract: Large language model (LLM) agents already trade real capital, and builders copy a few published designs that share models and news feeds, so...
arXiv:2610.28454v1 Announce Type: new Abstract: Agents are starting to sell to other agents. Seller agents write the listings of tools, services and products, and buyer agents read these listings whe...
The paper introduces a minimax framework to analyze the trade-off between model utility and vulnerability to distillation attacks, featuring an adaptive evaluation rule that reweights high-value examples and a teacher-side defense that suppresses outputs most useful for imitation. It proposes a cheap, forward-pass-only defense called Product-of-Experts (PoE) that combines the teacher with a proxy student during generation. Empirical results show a significant passive–adaptive gap, with adaptive students recovering more capability than passive evaluation suggests, narrowing the robustness gap between expensive defenses and PoE while keeping PoE cheaper and preserving higher-quality reasoning traces.
arXiv:2606. 05363v1 Announce Type: cross Abstract: On a platform with many sellers, should a pricing algorithm explicitly model competitors' prices when learning demand?
Lossless Anti-Distillation Sampling (LADS) is a defense that keeps the generation process unchanged while reducing the effectiveness of model distillation. It achieves this by coupling latent randomness across accounts, so that a single user experiences the same output as without defense, but a multi‑account distiller receives dependent data that hurts its generalization. Experiments on image, math, and code generation show that LADS degrades distilled model performance while preserving statistical fidelity for individual users.
The paper introduces Market Signal Injection (MSI), an attack that alters how market data is formatted or described—without changing its numerical values—to influence large language model (LLM) pricing agents. Experiments on nine open‑weight and three proprietary models in simulated duopoly and triopoly markets show that sentiment‑based formatting changes cause significant shifts in firm behavior, profits, and consumer surplus. The study also demonstrates that model susceptibility varies across families, that larger models are not always more robust, and that techniques such as input canonicalization and decision boundary anchoring can partially mitigate these attacks.
The poster presents a preliminary study on detecting whether a suspect language model was distilled from a proprietary teacher model or trained independently. The authors formulate distillation inference as a hypothesis test, using shadow models trained on either the teacher’s reasoning traces or reference answers to calibrate a p‑value for the suspect model’s predictions. In experiments with Qwen2.5‑7B as teacher and Llama‑3.2‑3B as suspect, the test achieved a true positive rate of 1.0 at a significance level of 0.02, showing the feasibility of detecting distillation attacks.
arXiv:2607. 26385v1 Announce Type: cross Abstract: Empirical work on algorithmic collusion asks one question of the data: are prices supracompetitive?