PLE: Physical-Engineering's Last Exam
Frontier language models now score above 90% on popular knowledge benchmarks and are increasingly deployed as agents that carry out multi-hour professional work. Yet no existing benchmark tests whethe...
Tool use, function calling, orchestration and the protocols that let models act rather than only answer.
Frontier language models now score above 90% on popular knowledge benchmarks and are increasingly deployed as agents that carry out multi-hour professional work. Yet no existing benchmark tests whethe...
LLM agents are often evaluated by reducing execution traces to labels such as final-answer correctness, tool verification, and fault propagation. These labels are measurements produced by parsers, beh...
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all existing paradigms: imi...
Today's multimodal assistants are typically prompted to interpret the visual evidence presented to them. To offload the cognitive burden of remembering across long horizons, wearable assistants should...
Recent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general-purpose AI models f...
Enterprise agents act on documents. Before an agent can follow a policy, answer a compliance question or file a ticket, some ingestion layer must turn a Word file or PDF into tokens - and that layer i...
This study introduces GuideEvolve, a self-evolving agent framework incorporating expert guidance and three autonomous regulatory mechanisms, collectively termed the 3Ts: Think Fast and Slow, Throwback...
LLM agents with similar success rates can follow different action sequences. An Agent Behavior Map represents LLM-harness combinations using centered action log-likelihoods on shared reference traject...
Reliable validation of autonomous driving cyber-physical systems requires testing not only nominal driving behavior but also failures arising from faults in sensing and embedded computation. Physical...
LLM-generated records increasingly populate production information systems where they are treated as data assets subject to the same quality expectations as records from operational databases. A singl...
Long-context continual pre-training (LCCP) is widely used to improve long-context capability, but how its benefits scale with the number of tokens used for LCCP remains unclear. We investigate this qu...
As LLM agents interact in the wild and shape each other's decisions, understanding how they converge toward agreement becomes important. We study this in multi-turn conversations where two LLMs discus...
User-facing LLM agents must identify which behavioral policies apply at each turn, but evaluating every policy with an LLM is costly. We introduce Signals and Queries (S&Q), a lightweight first-stage...
Patch the Planet is a platform that links security researchers, maintainers, and incident responders to collaboratively address critical vulnerabilities in open source software. By coordinating global response efforts, it aims to uncover and remediate vulnerabilities more efficiently. The initiative emphasizes the importance of community-driven security practices in the open source ecosystem.
Agentic search depends on the interface through which a language-model agent accesses and explores a corpus. Existing interface designs are fixed by hand and applied unchanged to every corpus, althoug...
Language-model agents are increasingly asked to run experiments, yet benchmarks rarely measure whether those experiments improve the decisions they are meant to inform. We introduce CrashGym, a determ...
Autonomous industrial inspection requires strictly deterministic real-time inference under tight memory and computational constraints. However, existing Video Instance Segmentation (VIS) models suffer...
OpenAI, PerturbAI, and Scripps Research employed distributed AI agents to analyze data from millions of brain cells, uncovering promising research leads. The collaboration leveraged thousands of AI agents to sift through vast biological datasets, identifying new insights that could advance neuroscience research.
arXiv:2610.27450v1 Announce Type: new Abstract: On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowled...
The paper introduces a runtime assurance layer for language‑model agents operating in operational workflows, addressing the evaluation–execution gap where approvals may become invalid after changes in facts, evidence, policy, or authority. The proposed state‑bound configuration separates probabilistic measurement from deterministic adjudication, binds execution allowances to the basis of an approval, invalidates them upon logged changes, and rechecks admissibility at dispatch. In a scripted 42‑case replay, the approach identified 15 later‑invalidated approvals that a non‑re‑evaluating comparator missed, holding 14 affected actions before dispatch and compensating one landed effect.