arXiv AI
Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jul 7
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.
Hugging Face Trending Papers
Jul 6
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining.
arXiv AI
Jul 28
TRE: Training-Free Hallucination Detection for Diffusion Language Models
arXiv:2607. 22661v1 Announce Type: new Abstract: Diffusion large language models (D-LLMs) have recently gained increasing attention, yet their reliability is significantly hindered by the hallucination problem.
arXiv Computation and Language
Oct 8
EASE: Entropy-Adaptive Distribution Shaping for Evading AI-generated Text Detectors
arXiv:2610.09976v1 Announce Type: new Abstract: AI-generated text (AIGT) detection can be sensitive to the decoding choices of the source large language model (LLM). We observe that perturbing next-t...
arXiv Computation and Language
Aug 27
Unveiling Spectral Mechanisms in Training-Free LLM Text Detection
The paper investigates training‑free detection of machine‑generated text using spectral analysis. It shows that spectral energy correlates with variance in token probability trajectories and that human writing produces characteristic fluctuations, termed "generative vitality." The authors find that spectral signals are strongest for long, continuous, constrained generations, while shorter or mixed texts require additional confidence‑based metrics.
arXiv AI
Jul 17
Tracing LLM Behavior to the Training Data with Empirical Next-Token Distributions
arXiv:2607. 14306v1 Announce Type: new Abstract: In this paper, we study the connection between an LLM's output distribution and the data used to train it.