arXiv Machine Learning

Stability-Aware Feature Design for Robust Watermark Detection in Machine-Generated Text

The paper introduces Pattern Stability Score (PSS), a watermark detection framework that uses local statistical features and stability dynamics across paraphrased variants to identify machine-generated text. PSS combines global and local z‑score features with higher‑order run‑length statistics, autocorrelation signals, and stability scores over paraphrase depth. Experiments on PG‑19, CNN/DailyMail, and WikiText with Llama‑3‑8B, Qwen2‑7B, and multiple paraphrasers show that PSS improves detection AUC by 10‑15 percentage points and a single universal classifier achieves over 87.8% AUC across diverse LLMs, paraphrasers, and domains without retraining.

arXiv Machine Learning
2d ago

Dataset Watermarking with Provable Black-Box Detection

The paper introduces a dataset watermarking technique that embeds a watermark by increasing the co‑occurrence of randomly selected word pairs through meaning‑preserving local edits. The watermark can be detected solely from generated text with provable false‑positive control, and experiments on four base models and three datasets show reliable detection (p < 0.01) even when the watermarked data constitutes less than 5% of fine‑tuning tokens. Compared to existing methods, the approach better preserves benchmark utility and semantic integrity.

By Pengrun Huang, Kamalika Chaudhuri, Yu-Xiang Wang
arXiv AI
Sep 25

R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection

R-DEIM Net is a 76‑million‑parameter dual‑expert model designed for paraphrase detection that balances accuracy with computational efficiency. It combines an Interaction Expert, which captures token‑level similarity via multi‑scale 2D convolutions and attention, with a Reasoning Expert that generates human‑readable rationales using a Flan‑T5‑small decoder. On the Quora Question Pairs dataset, the model attains 90.07% accuracy and 90.16% F1‑score, matching strong transformer baselines while producing auxiliary rationales.

By Pushp, Vaibhav Prajapati, Himangshu Sarma
arXiv Machine Learning
Aug 28

When Is Noise Response Universal? Tokenization as the Hidden Variable in Language Models

The study investigates how textual neural models degrade when inputs contain noise such as typos, OCR errors, or dropped words. It finds that model performance decline is largely consistent across architectures under word‑level noise but diverges under character‑level noise, a difference attributed to tokenization rather than architecture. By applying a short contrastive training recipe, diverse encoders converge to a common robustness curve, enabling prediction of a model’s noise resilience and the ability to enhance robustness at specific noise scales through targeted training.

By Yefan Tao, Gerald Friedland, Luyang Kong
arXiv Computation and Language
Aug 31

Beyond Global Scalars: Synergizing Token-Level Statistics and Deep Semantics for Adversarial AIGC Text Detection

The paper introduces MOSAIC, a large adversarial benchmark for detecting AI-generated text, and presents NeuroStat, a new framework that combines token‑level probabilistic logits with deep semantic hidden states from a single language model. NeuroStat fuses these signals via Macro‑State Residual Modulation and uses orthogonal and contrastive losses to learn complementary representations. Experiments show that NeuroStat outperforms existing methods on MOSAIC, achieving superior robustness against adversarial attacks.

By Peiming Li, Yifan Wang, Zhiyuan Hu, Shiyu Li, Zheng Wei, Yang Tang
arXiv Computation and Language
2d ago

From Pixels to Pairs: A Comprehensive Benchmark of LLM-Driven Key-Value Extraction in Noisy Document Settings

The paper introduces a controlled benchmark for evaluating large language models (LLMs) on key‑value pair extraction from documents with varying levels of OCR noise. It tests 136 configurations across five instruction‑tuned open‑weight LLMs, three datasets, and four text‑quality conditions, using deterministic decoding to generate 17,688 document‑level inferences. The study finds that clean‑text performance does not reliably predict real‑world robustness, model rankings can reverse under noisy conditions, and few‑shot demonstrations do not always improve accuracy, highlighting reliability risks in OCR‑to‑LLM pipelines.

By Zahra Anvari