arXiv Computation and Language
Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 24
Experts Rise Where LLMs Disagree: Using Cross-Model Disagreement to Target Expert Effort in LLM Codebook Revision for Large-Scale Annotation
The paper proposes using large language models (LLMs) to identify disagreements among models as a way to focus expert effort on revising codebooks for large‑scale text annotation. Three expert feedback methods are evaluated: editing LLM‑generated revisions (Codebook Verifying), answering questions about disagreements (Question Answering), and labeling disagreement cases with rationales (Rationale Labeling). Experiments on tutoring‑session transcripts show that Rationale Labeling achieves the highest LLM‑labeling accuracy (64.9%) compared to the expert‑revised codebook (57.8%), with Question Answering also outperforming the baseline (60.5%).
arXiv Computation and Language
3d ago
Taming Speculative Search for Test-Time Scaling in LLM Serving
arXiv:2609.39334v1 Announce Type: cross Abstract: Test-time scaling has recently emerged as a powerful approach for improving LLM reasoning by allocating additional computation during inference, subs...
arXiv Machine Learning
Jun 5
Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation
arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.
arXiv AI
Aug 11
Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
arXiv:2608. 08512v1 Announce Type: new Abstract: Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates.
arXiv AI
Aug 24
PromptResponse: Optimizing Prompts for LLM Coding Tasks
arXiv:2608.21074v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to in...
arXiv AI
Jul 15
Rethinking Reward Models for Multi-Domain Test-Time Scaling
arXiv:2510. 00492v3 Announce Type: replace Abstract: The reliability of large language models (LLMs) during test-time scaling is often assessed with \emph{external verifiers} or \emph{reward models} that distinguish correct reasoning from flawed logic.