The paper introduces BoardSafety, a benchmark that transforms manufacturer datasheets and hardware guides into a board‑specific safety rule base for evaluating large language models (LLMs) on hardware safety judgments. Each data item pairs a development board and a proposed operation with a safety rule and a ground‑truth label (safe, unsafe, or insufficient‑info). Experiments show that providing safety rules improves model accuracy, while structured‑rationale prompting yields mixed results, indicating that current models cannot reliably assess operation safety.
arXiv:2607. 29254v1 Announce Type: new Abstract: AI agents extend large language models (LLMs) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real-world actions.
arXiv:2508.15526v2 Announce Type: replace Abstract: The rapid proliferation of large language models (LLMs) has intensified the requirement for reliable safety evaluation to uncover model vulnerabili...
arXiv:2608. 14590v1 Announce Type: new Abstract: LLM agents increasingly perform irreversible real-world actions, including database updates, API calls, file operations, and autonomous use of tools.
SAFARI is the first industrial benchmark for evaluating large language models (LLMs) in automotive hazard analysis and risk assessment (HARA) under ISO 26262. It comprises 3,000 de‑identified HARA cases and tests two tasks: open‑ended hazard generation and standards‑grounded risk classification, using a novel reference‑anchored LLM‑as‑a‑judge protocol. Experiments with nine state‑of‑the‑art LLMs show that while hazard narratives are often plausible, risk classification remains weak (best ASIL macro‑F1 = 0.261), with errors mainly due to missing scenario context and misjudged controllability. "whyItMatters":"The benchmark highlights the current limitations of LLMs in safety‑critical engineering workflows, guiding future research and expert oversight in automotive safety analysis."
arXiv:2606. 03648v1 Announce Type: cross Abstract: Adapting foundation large language models to a user's task or preferred style through fine-tuning can result in compromising the model's safety.
arXiv:2608. 12025v1 Announce Type: cross Abstract: Medical devices are becoming more software-intensive, connected, and AI-enabled.
The paper investigates whether existing AI safety benchmarks, designed for large language models, are suitable for evaluating small language models (SLMs). By testing five benchmark suites on 26 open‑source SLMs with a unified scoring rubric, the authors find that ambiguous judgments dominate, especially for complex prompts and certain architectures. This ambiguity, linked to factors like lexical density and output perplexity, undermines the reliability of aggregate leaderboards and reveals a confound between model capability and perceived safety.
arXiv:2610.35671v1 Announce Type: new Abstract: As frontier language models grow increasingly capable and autonomous, independent third-party evaluations are critical for characterizing their systemi...
Safin-1.5 is a large language model that incorporates a safety‑native architecture and a recursive safety improvement loop to enhance its own safety while maintaining utility. The architecture uses persistent safety states and hazardous‑knowledge disentanglement to isolate and remove risky knowledge with minimal impact on general capabilities. An inference‑time harness extends the model’s abilities to formal reasoning by combining language understanding with symbolic inference and verification, enabling the construction of checkable proofs.
TRACE is a new benchmark that evaluates the safety of Large Reasoning Models (LRMs) across the entire inference pipeline, including prompts, reasoning traces, and final responses. It provides prompts in two languages covering nine risk categories and ten attack strategies, and for each prompt four LRMs generate traces and responses that are annotated for safety with supporting evidence extracted from the source text. Evaluation of 18 guardrail models on TRACE shows that detecting unsafe content in reasoning traces is much harder than in prompts or final responses, and that current models struggle to extract the necessary evidence.
arXiv:2606. 16808v1 Announce Type: new Abstract: While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries.
arXiv:2602.03100v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguisti...