arXiv:2608. 09093v1 Announce Type: cross Abstract: How a document's arrangement is written down, its notation, is a training variable that no dataset card records.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2609. 31181v1 Announce Type: new Abstract: Black-box model identification works by scoring a model's response to natural-language prompts.
The paper investigates how much a masked‑text reconstruction predictor’s outputs would differ if a second corpus were collected from the same population. Using a smoothed predictor based on token‑pair counts, the authors derive a first‑order formula for expected disagreement between predictors trained on two independent corpora, accounting for document length, random context counts, and token‑offset correlations. Experiments on TinyStories, WikiText‑103, and CNN/DailyMail show that increasing corpus size from 512 to 2,048 chunks of 64 tokens reduces disagreement only modestly (5–40 %), and that forecast accuracy improves with larger reference corpora and a source‑split jackknife correction. whyItMatters:"The study provides a practical method to estimate how much predictor outputs would change with additional data, informing decisions about data collection and model robustness in masked‑text reconstruction tasks."
arXiv:2608. 00144v1 Announce Type: new Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines separate members from non-members from surface text alone.
The paper reports on a deployed multi‑agent tender‑response system that uses an open‑weights language model under sovereignty constraints. In a blind comparison, the system’s answers were judged at least as good as human‑written bids in 40 of 55 sections, with only a few gaps attributable to missing knowledge rather than writing quality. The study also demonstrates an asymmetry in conditioning: while structural markup improves reading tasks, converting instruction material from prose to nested XML degrades answer quality, and naming forbidden constructions concentrates defects.
The paper investigates whether language models can identify sentences from their training data by using exact duplication counts from publicly released corpora for two model families, OLMo‑2 and Pythia. It finds that for typical duplication levels, models show only a weak trace of exposure, with a rank correlation near –0.08, and that strong signals only appear when a sentence appears roughly a thousand times, at which point fame rather than memory dominates. The study also demonstrates that common membership tests can be misleading, as changing a single word does not alter the model’s preference, and that controlling for register can significantly improve detector performance.
arXiv:2608. 14509v1 Announce Type: new Abstract: Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt.