arXiv:2503. 14499v4 Announce Type: replace Abstract: Despite rapid progress on AI benchmarks, the real-world meaning of benchmark performance remains unclear.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2606. 07157v1 Announce Type: new Abstract: Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning.
arXiv:2602. 07267v2 Announce Type: replace Abstract: Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty.
ASI‑Bench is a new benchmark that evaluates AI systems on their ability to conduct innovative exploration and autonomous scientific research across 11 domains, using 60 project‑level tasks. It progressively removes human methodological guidance to test whether AI can independently select methods, execute research, and produce verifiable results. Results from 18 state‑of‑the‑art agent–model configurations show a sharp performance drop when guidance is reduced, indicating current systems still rely heavily on human input.
arXiv:2606. 05080v1 Announce Type: new Abstract: Scientific and engineering progress is fundamentally a long-horizon iterative process: proposing changes, running experiments, measuring outcomes, and continuously refining artifacts.
arXiv:2502. 20502v2 Announce Type: replace Abstract: Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated data, are increasingly posited as approximate models of human cognition.
arXiv:2606. 30182v1 Announce Type: new Abstract: AI models are rapidly improving at autonomous coding, as shown by benchmark progress and one-off demonstrations such as AI implementing a C compiler.