arXiv AI
Jul 28
Epistemic Norms for AI Safety and Alignment Research
arXiv:2607. 24243v1 Announce Type: new Abstract: Mainstream AI research emphasises capability growth and tolerates low failure rates when average-case performance is high.
The safety failures we are not instrumenting: a perspective on hidden safety-critical challenges in modern AI systems
arXiv:2607. 19292v1 Announce Type: cross Abstract: Current AI safety discourse still focuses disproportionately on visible failures, including obvious harms, dramatic misuse, and hypothetical catastrophic scenarios.
Related stories
arXiv AI
Jul 2
From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems
arXiv:2410. 22526v2 Announce Type: replace Abstract: To effectively address potential harms from Artificial Intelligence (AI) systems, it is essential to identify and mitigate system-level hazards.
arXiv AI
3d ago
A Competing-Hazards Systematization of Loss of Control in Autonomous Agents
arXiv:2609.38411v1 Announce Type: new Abstract: Leading AI developers have reported agents acting beyond their approved limits, which a United Nations panel described as an early warning of loss of h...
arXiv AI
Sep 15
AI Deployment Accountability Engineering: A Vision for Accountable AI in Safety-Critical Socio-Technical Systems
The paper proposes AI Deployment Accountability Engineering (ADAE), a new subdiscipline focused on establishing measurable, continuous, and actionable accountability for AI systems once they are deployed. ADAE treats accountability as a deployment-layer property, aiming to ensure systems remain within acceptable risk limits, identify failure contexts, attribute failures across technical and human components, and translate technical failures into downstream consequences. The authors outline a research agenda built around four pillars—structured discovery of context-dependent failure modes, privacy-preserving accountability measurement, system-level risk analysis for agentic AI, and translation of technical failures into operational and institutional risks—to support timely intervention in safety-critical socio-technical environments.
arXiv AI
Aug 18
Position: AI Lock-In Is in Progress, and We Must Be Prepared
arXiv:2608. 14565v1 Announce Type: new Abstract: AI safety research has mainly focused on two areas: technical alignment (ensuring AI systems produce human-aligned outputs) and the regulation of generative AI's societal impacts (including unemployment risk and labor market disruption).
arXiv AI
Aug 13
Agent Safety Should Be a Runtime Contract
arXiv:2608. 11274v1 Announce Type: cross Abstract: The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI.
arXiv AI
Aug 17
Why we need an AI-resilient society
arXiv:1912. 08786v3 Announce Type: replace-cross Abstract: Three generations of software have transformed the role of artificial intelligence in society.
arXiv AI
Aug 25
Why we need an AI-resilient society- Profiling Large Language Models
The article discusses the evolution of AI across three generations—from explicit logic to neural networks to large language models (LLMs)—and how LLMs introduce new systemic risks. It applies a forensic‑psychology profiling method to identify ten key features of LLMs, such as hallucinations, bias, and cognitive atrophy, revealing an entity that confabulates, amplifies user biases, and erodes human competence. The report concludes with a four‑pillar framework for AI resilience, emphasizing cognitive sovereignty, measurable control, partial autonomy, and openness to safeguard society.
arXiv AI
Aug 26
Quantifying System-Level Harms from AI Adoption in Complex Sociotechnical Systems
The paper proposes a framework that connects structured hazard analysis, component-level testing, and probabilistic system modelling to assess system-level harms from AI in complex sociotechnical systems. It demonstrates the approach using the UK's Real Time Gross Settlement system, showing how adversarial inputs to LLM-based trading can shift AI behaviour, reduce system resilience, and increase the likelihood of cascading bank failures. The framework aims to provide a traceable pathway from model behaviour to systemic outcomes, enabling evidence-based governance of AI in critical infrastructure.
arXiv AI
Jul 22
From Agent Failure Paths to Quantified Residual Risk: A Compositional Framework for Resilient Agentic AI
arXiv:2607. 18243v1 Announce Type: new Abstract: Agentic AI is crossing trust boundaries faster than current risk models can represent.
arXiv AI
Aug 11
Unaccountable Delegation, Fading Skills: Mapping the Risks of Workplace AI Agents
arXiv:2608. 08601v1 Announce Type: new Abstract: To anticipate socio-technical risks from AI agents, organizations need taxonomies to classify them.
arXiv AI
Sep 25
The Gold in Bias: Maturing the AI Design Process through Verification
The paper proposes rethinking bias in AI as a diagnostic tool rather than merely a flaw to be minimized. It introduces a multidimensional framework that examines bias across origin, lifecycle emergence, technical causes, and validation methods, covering 30 bias types, 16 verification methods, and 20 countermeasures for both traditional and generative AI. The authors present a hierarchical evidence framework distinguishing internal and external validity, and advocate for Ethics by Design principles to embed bias verification throughout the AI development lifecycle.