arXiv AI
Self-Reports Are Not Verification: Environment-Grounded Auditing of LLM Operators in Evolutionary Search
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
3d ago
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of magnitude more interactions than any evaluation...
arXiv AI
3d ago
BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing
arXiv:2608.31105v1 Announce Type: new Abstract: Users of a deployed language model routinely encounter behaviours that testing almost never surfaces, since deployment puts the model through orders of...
arXiv AI
Aug 17
ASSERT: A Measurement Pipeline for GenAI Audits
arXiv:2608. 13840v1 Announce Type: cross Abstract: Audits of generative AI (GenAI) systems often summarize behavior as a reported rate: how often the audited system complies with policy.
arXiv AI
Jun 30
SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution
arXiv:2606. 29713v1 Announce Type: cross Abstract: Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit.
arXiv Machine Learning
Jul 1
White-Box Sensitivity Auditing with Steering Vectors
arXiv:2601. 16398v3 Announce Type: replace-cross Abstract: Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators.
arXiv Computation and Language
3d ago
S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?
S3Gym is an interactive benchmark designed to evaluate large language models (LLMs) on their ability to self-improve through self-testing, self-judging, and self-improvement. It separates permissive exploration from strict held-out evaluation across seven text-based games with executable environment verifiers. Experiments show that self-improvement varies by task, with different experience incorporation pathways (direct history, summary memory, or parameter training) yielding mixed results and highlighting the need for agents to transform feedback into executable, transferable policies.