Governance of superintelligence
Now is a good time to start thinking about the governance of superintelligence—future AI systems dramatically more capable than even AGI.
arXiv:2412. 16468v4 Announce Type: replace Abstract: The emergence of large language models (LLMs) has sparked discussion on Artificial Superintelligence (ASI), a hypothetical AI system that surpasses human intelligence.
Now is a good time to start thinking about the governance of superintelligence—future AI systems dramatically more capable than even AGI.
AI agents are increasingly autonomous, posing significant risks that current designs hinder effective human oversight. The paper argues that oversight is degraded by both design choices and the cognitive decline of users who rely heavily on automation. It calls for prioritizing human cognitive needs in AI agent development, proposing design affordances and protocols to maintain critical judgment and counter skill atrophy.
arXiv:2606. 03237v1 Announce Type: new Abstract: AI's central challenge is shifting from capability to coexistence.
arXiv:2606. 12683v1 Announce Type: new Abstract: Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations.
arXiv:2606. 30481v1 Announce Type: cross Abstract: Current large language models are extraordinary statistical engines.
arXiv:2609.05894v1 Announce Type: new Abstract: The exponentiation of Artificial intelligence (AI) in the recent past has entered a transformative era that has been driven by the growth in large lang...
arXiv:2510. 26518v2 Announce Type: replace Abstract: Human feedback is critical for aligning AI systems to human values.
ASI‑Bench is a new benchmark that evaluates AI systems on their ability to conduct innovative exploration and autonomous scientific research across 11 domains, using 60 project‑level tasks. It progressively removes human methodological guidance to test whether AI can independently select methods, execute research, and produce verifiable results. Results from 18 state‑of‑the‑art agent–model configurations show a sharp performance drop when guidance is reduced, indicating current systems still rely heavily on human input.
arXiv:2606. 05770v1 Announce Type: cross Abstract: AI is changing how software engineers work, but it often comes with hidden burdens and costs.
The paper introduces Atria Dawn Preview, a foundation agentic language model aimed at scientific research and engineering workflows. Trained through a Verifiable Experience Pipeline, it performs competitively across 16 real‑world benchmarks, achieving the highest scores on five. The authors also present a detailed case study of human–AI collaboration, showing that while AI proposes methods and revisions, humans retain final decision‑making and guide the research direction.
arXiv:2608. 12320v1 Announce Type: cross Abstract: This article reviews and updates the framework for accountability in AI based on account- ability ecosystems.
The advancement of AI capabilities compels researchers and the public to be more aware of its potential worldwide impact. A pressing near-term concern is the regulation of military AI applications.