The study investigates how many synthetic conversation samples are required to train activation probes that monitor language models for high‑stakes, harmful, and instruction‑following behaviors. By tracking learning curves across 10–590 samples on various probe models, generators, and prompts, the authors find that the number of samples needed is largely determined by coverage of the monitored concept rather than the difficulty of individual sample types. They show that high‑stakes and harmful probes reach near‑plateau performance with as few as 80 samples, while instruction probes require several times more, and that breadth of sample kinds is more critical for instruction monitoring than for high‑stakes scenarios. The paper also releases evaluation suites, dev sets, and generated data for further research.
arXiv:2610.35586v1 Announce Type: new Abstract: Linear probes, classifiers that read a language model's internal activations, are widely used to detect harmful requests. They are cheap to train and r...
arXiv:2606. 28471v1 Announce Type: new Abstract: Model capability is the central variable in LLM pre-training, yet is never observed directly: data shapes it prospectively, while evaluation reveals it only retrospectively, compressing samples, prompts, decoding, and scoring rules into one noisy score.
arXiv:2608. 10441v1 Announce Type: new Abstract: Many pipelines can pay a per-example cost to acquire an auxiliary, model-derived observation -- an LLM's structured reasoning, a slow oracle, an expensive measurement -- and then must decide when the acquired signal is worth using.
arXiv:2607. 18305v1 Announce Type: cross Abstract: Some limits on what language models know are not gaps in data coverage but structural properties of learning from text.
The paper investigates whether adding a short classification instruction after a user’s prompt improves the ability of activation probes to detect malicious inputs in large language models. Across 13 safety benchmarks and three open‑weight model families, a classification suffix consistently boosts out‑of‑distribution detection (up to ~4 AUC points) compared to no suffix, and the benefit transfers to multi‑position pooling probes used in production. The improvement stems from the classification format itself rather than the specific content of the instruction, though the optimal suffix varies with the model and readout type.
The paper introduces two scaling laws that predict the performance of safety probes—tools used to monitor language models for harmful behavior—based on model size and the number of labeled examples. By creating six large safety datasets and training probes across multiple model families, the authors show that probe error decreases as a power law with model size and further improves with more examples, with the improvement rate increasing for larger models. These laws enable researchers to forecast probe accuracy and decide whether to invest in more data or a larger model to achieve a desired error threshold.
arXiv:2607. 09709v1 Announce Type: new Abstract: Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact.
arXiv:2608.28021v2 Announce Type: replace-cross Abstract: Large language models increasingly author Infrastructure-as-Code (IaC), where one insecure default is provisioned straight into production. P...
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.
arXiv:2608. 13087v1 Announce Type: cross Abstract: Neural combinatorial optimization (NCO) solvers report the best of many sampled solutions per instance, and the sample count is, by convention, identical for every instance.
arXiv:2607. 25292v1 Announce Type: new Abstract: Silicon sampling uses language models as proxies for human survey respondents, treating each model call as an independent draw from the persona's response distribution.
arXiv:2604. 24827v2 Announce Type: replace-cross Abstract: Closed-source frontier labs do not disclose parameter counts.