arXiv AI By Haiyan Zhao, Zirui He, Yiming Tang, Fan Yang, Ali Payani, Dianbo Liu, Mengnan Du

Rep2Text: Decoding Full Text from a Single LLM Token Representation

Read the original on arXiv AI →

The paper introduces Rep2Text, a method that reconstructs text from a single hidden representation of a large language model. Experiments show that for short passages (~16 words) about half the words can be recovered while preserving overall meaning, and even longer texts retain recoverable main ideas. These results suggest that hidden representations encode more information than previously thought, raising concerns about privacy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Oct 19

GPT, But Backwards: Exactly Inverting Language Model Outputs

The paper introduces Sparse One-hot Discrete Adam (SODA), a white‑box search algorithm that can reconstruct textual inputs from partial knowledge of a language model’s output logits. SODA can recover natural sequences of up to 17 tokens with no false positives, outperforming prior methods by a factor of four in exact‑match reconstruction. The authors demonstrate that input length and vocabulary size are more critical to inversion difficulty than model size, and provide proof‑of‑concept security applications, including recovering backdoor triggers and personally identifiable information from a poisoned LLM and synthetic user‑chatbot interactions.

By Adrians Skapars, Edoardo Manino, Youcheng Sun, Lucas C. Cordeiro
arXiv Computation and Language
Sep 7

Robust Text Watermarking for Large Language Models via Dual Semantic Embeddings

The paper introduces Dual-Embedding Watermarking (DEW), a semantic watermarking technique for large language models that combines contextual and token-level embeddings. DEW applies algebraic vector-space operations to generate a watermark signal that remains robust to paraphrasing and translation, while obfuscating the signal with pseudo-random matrices seeded by a secret key. Experiments demonstrate state‑of‑the‑art robustness, especially against translation, with minimal computational overhead and preserved text quality at lower watermark strengths.

By Jonas Sch\"afer, Cezary Pilaszewicz, Gerhard Wunder