AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv AI
Sep 17

Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations

The study evaluates gender representation in 8,000 images generated by four generations of the Stable Diffusion text‑to‑image model across 20 occupations and five prompt templates. It finds that 76.4% of the images depict male subjects, with 57.6% of historically female‑coded occupations also showing male subjects, and that newer model generations do not consistently reduce bias. Compared to U.S. Bureau of Labor Statistics data, the models underrepresent women by 20–46 percentage points, especially in near gender‑balanced fields such as scientists and cleaners.

By Shesh Narayan Gupta, Nik Bear Brown
arXiv AI
Sep 17

DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning

DualSQL is a Text-to-SQL system that uses two agents sharing a single model backbone, enabling joint optimization via multi-agent reinforcement learning. The approach incorporates three database access tools for multi-step reasoning, rollout guardrails to stabilize training, and a new SQL correctness metric called robust execution match (REX). Trained on only 3,755 examples, DualSQL-4B reaches 68.0% execution accuracy on the BIRD dev set, while DualSQL-8B achieves 71.1%, surpassing prior state‑of‑the‑art single‑model solutions with 32B parameters.

By Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li, Sravan Babu Bodapati, Cody J. Greer, Yu Su, Fatma Ozcan
arXiv AI
Sep 17

On-the-Fly Homographies Calibration for Multi-Camera Tracking

The paper introduces an on-the-fly homography calibration system for multi-camera tracking that starts from coarse manual homographies and refines them using a centroid-based projection optimization (PO) on live detection metadata. PO continuously aligns ground-plane geometry without adding computational latency, enabling the system to adapt automatically to camera movements or environmental changes. The refined geometry feeds a bird's-eye-view tracker that fuses detections and unifies trajectories across zones while maintaining privacy safety and zero overhead.

By David Voihanski, Mor Sinai, Ben Zion Bobrovsky
arXiv AI
Sep 17

Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation

The paper introduces Label-Confidence-Aware Uncertainty Quantification (LCA-UQ), a method that uses Pointwise Kullback-Leibler divergence to align global entropy from multiple stochastic samples with the local confidence of a candidate answer. By bridging this gap, LCA-UQ improves the reliability and stability of uncertainty assessments in natural language generation. Experiments on popular LLMs and NLP datasets show that label sources significantly influence classification and that LCA-UQ outperforms existing uncertainty estimation approaches.

By Qinhong Lin, Yinglun Feng, Yuhao Zhang, Zhongliang Yang, Linna Zhou
arXiv AI
Sep 17

Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism

The paper introduces ED$^2$, an end‑to‑end large language model–based sequential recommender that integrates index generation and recommendation through a dual dynamic index mechanism. It employs a multigrained token regulator for alignment supervision and custom instruction‑tuning tasks to capture high‑order user‑item interactions. Experiments on three public datasets show ED$^2$ outperforms baselines with an average 19.62% gain in Hit‑Rate and 21.11% in NDCG.

By Jun Yin, Zhengxin Zeng, Mingzheng Li, Hao Yan, Chaozhuo Li, Weihao Han, Jianjin Zhang, Ruochen Liu, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang, Shirui Pan, Senzhang Wang
arXiv AI
Sep 17

CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art

CompArt introduces a new approach to aesthetic alignment in text-to-image generation by using the Principles of Art (PoA) such as Balance, Rhythm, and Emphasis to define explicit compositional constraints. The authors create a large dataset of 80,032 WikiArt images, each annotated with PoA analyses generated by a multimodal LLM, and present ArtDapter, a lightweight adapter that steers a pretrained diffusion model along ten PoA dimensions while preserving semantic fidelity. Experiments demonstrate that CompArt outperforms strong baselines in adhering to PoA controls under a dual evaluation protocol.

By Zhe Jin, Tat-Seng Chua
arXiv AI
Sep 17

SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis

SurgRAW introduces a multi‑agent, chain‑of‑thought workflow for robotic surgical video analysis, leveraging a new SurgCoTBench benchmark with 14,256 QA pairs across five surgical tasks. The system uses an orchestrator to split scene understanding into two reasoning streams, panel‑discussion‑style collaboration among task‑specific agents, and retrieval‑augmented generation to incorporate surgical knowledge. Experiments show SurgRAW outperforms mainstream vision‑language models and a supervised baseline by 14.61% accuracy.

By Chang Han Low, Ziyue Wang, Tianyi Zhang, Zhu Zhuo, Zhitao Zeng, Evangelos B. Mazomenos, Yueming Jin
arXiv AI
Sep 17

A visual large language foundational model for medical image recognition using clinician-contributed online resources

The paper introduces ThoughtMed-1M, a large-scale medical visual question answering dataset built from de‑identified images and clinician‑generated commentaries, designed to capture structured clinical reasoning and image‑text alignment. Using this dataset, the authors train FOLTMed, a foundational large language model that achieves state‑of‑the‑art performance on 42 medical VQA benchmarks, with a macro accuracy of 85.4% and improved factuality and similarity metrics over existing models.

By Lingxuan Hou, Yuhua Xie, Yue Hu, Yan Zhuang, Junqi Li, Chengzhi Xia, Binh Phu Nguyen, Abubakar Siddique, Minh Nguyen, Yao Hou, Yanju Bao, Kexin Liu, Ke Chen, Jianjun Sun, Zeqi Li, Trung Nguyen, Jiangli Lin
arXiv AI
Sep 17

Language-Guided Terrain-Adaptive Neural MPC for Autonomous Traversal of Articulated Tracked Robots

The paper introduces ASTRIL-MPC, a language‑guided neural model predictive control framework that enables articulated tracked robots to navigate complex, contact‑rich urban environments such as stairwells and cluttered interiors. By combining a learned kinematics model that predicts short‑horizon state changes, an optimization‑based planner with multi‑objective costs, and a large language model that safely updates control weights, the system achieves up to 71% better traversal quality than non‑adaptive NMPC and 67% better than a PPO baseline, while eliminating collision impacts during descent. Real‑robot trials over four indoor obstacles confirm the method’s transferability to physical contact‑rich traversal.

By Zhenfeng Gan, Yanbo Chen, Lirong Che, Yongyi Ma, Rongkai Zhu, Xueqian Wang
arXiv AI
Sep 17

Ultralytics YOLO Evolution: An Overview of YOLO27, YOLO26, YOLO11, YOLOv8, and YOLOv5 Object Detectors for Computer Vision and Pattern Recognition

This paper provides a detailed overview of the Ultralytics YOLO family from YOLOv5 to YOLO27, highlighting key architectural changes, benchmarking results, and deployment considerations. It discusses the evolution of each version—YOLO27’s scale‑adaptive dual architecture, YOLO26’s loss and optimization refinements, YOLO11’s efficiency focus, YOLOv8’s anchor‑free detection, and YOLOv5’s modular ecosystem—alongside performance metrics on COCO and latency on TensorRT. The review also surveys applications in robotics, agriculture, surveillance, and manufacturing, and outlines future challenges such as dense scene handling, CNN‑Transformer integration, and hardware‑aware optimization.

By Ranjan Sapkota, Manoj Karkee
arXiv AI
Sep 17

Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI

The paper proposes an independence‑graded audit protocol for agentic AI systems, arguing that independence should be evaluated along three orthogonal axes: principal independence, substrate independence, and evidence independence. It introduces a seven‑step protocol based on the beta‑factor model from reliability engineering, demonstrates its application through a structural detectability analysis and a Monte Carlo study, and maps the framework to relevant regulatory standards such as the EU AI Act, ISO/IEC 42006, and UK public‑sector guidance.

By Mohamed Chahine Ghanem
arXiv AI
Sep 17

REPAIR: Resolving Long-Tail Confusion in Scientific Retrievers via Fact-Verified Iterative Refinement

REPAIR is a self‑evolving data augmentation framework designed to improve scientific dense retrievers by addressing long‑tail concept gaps and fact sensitivity. It iteratively generates training data through diagnosis of long‑tail concepts, API‑guided evidence expansion, and hard negative mining, thereby grounding retrieval in factual reality. Experiments show that REPAIR outperforms 19 strong baselines across nine materials science and biomedical benchmarks.

By Yerim Oh, Gunhee Kim
arXiv AI
Sep 17

Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders

The paper investigates how role assignment in large language model (LLM) recommenders influences sponsorship bias. By assigning the agent’s principal as either a traveler or a booking platform, the authors find that platform delegation reduces the penalty applied to sponsored listings and weakens consumer skepticism triggered by disclosure. The study also shows that stricter terminology and attribution to the platform widen the divergence in agent evaluations, indicating that current disclosure mandates are insufficient to protect consumers in AI-mediated commerce.

By Davood Wadi, Yu Ma
arXiv AI
Sep 17

RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos

RoboVAD is a large-scale benchmark for video anomaly detection in robotic arm manipulation, featuring cross-domain evaluation where both tasks and anomaly types are unseen during training. The dataset challenges existing VAD methods, with state-of-the-art approaches, including a new method tailored for robotic arms, still achieving less than 70% micro-averaged frame-level AUC in the hardest setting. The authors provide the dataset and code publicly for further research.

By Alexandru-Bogdan Dura, Sebastian Balmus, Radu Tudor Ionescu
arXiv AI
Sep 17

I code or AI code: A comparative evaluation of AI-rated scores in classroom observations

The study evaluated whether a large language model (GPT‑5) could score teacher‑child interactions in early childhood classrooms using the Classroom Assessment Scoring System (CLASS) framework, comparing its results to human raters. Across 87 video‑recorded observations from 38 classrooms in Hong Kong, AI scores converged most closely with human ratings in the Emotional Support domain, especially the Quality of Feedback dimension, while diverging more in procedural or context‑dependent areas such as Classroom Organization and Instructional Support. The findings suggest that transcript‑based AI scoring can serve as a preliminary screening tool to aid teacher reflection, but it is not yet reliable enough to replace trained observers for full CLASS evaluations.

By Y. Fong, J. Xiang, T. Y. D. Chan, K. Lee, E. Y. H. Lau
arXiv Machine Learning
Sep 17

Revisiting Distributed Sign-Based Variance Reduction

The paper addresses bias introduced by aggregating local signs in distributed sign-based variance reduction methods, which hampers optimal convergence rates. By proposing an unbiased compression of recursive gradient increments to track the global gradient at the server, the authors achieve optimal convergence rates for both nonconvex stochastic and finite-sum optimization. They provide specific rate bounds for α-norms and demonstrate matching sample complexities to centralized settings for finite-sum problems.

By Wei Jiang, Zechao Li, Lijun Zhang
arXiv AI
Sep 17

Prepared Or Unprepared? Evaluating Healthcare Workforce Readiness for Clinical Adoption of Artificial Intelligence in Nigeria

The study surveyed 761 Nigerian healthcare professionals to assess their readiness for AI adoption in clinical settings. While 92.6% were aware of AI, only 63.0% felt prepared and 40.9% reported low knowledge, highlighting a gap between awareness and readiness. Major barriers identified were lack of training, poor infrastructure, high costs, job displacement fears, ethical and data privacy concerns, with significant regional and professional differences in preparedness and attitudes.

By Abbas M. Rabiu, Abdulrazaq A. Zubair, Um-mulkhairi Ibrahim, Tolulope Olusuyi, Shaheeda Farouq, Safwan M. Dafi, Adaobi C. Emegoakor, Yewande Gbadamosi, Maruf Adewole
arXiv Machine Learning
Sep 17

WaveTLM: Reliable Time-Series Language Modeling through Task Compilation

WaveTLM introduces a unified compiler‑executor framework that transforms natural‑language requests into typed task states and reliable numerical or structured outputs for time‑series tasks. The authors present ExecTS‑QA, a benchmark covering forecasting, imputation, classification, anomaly detection, and waveform analysis, and show that a single WaveTLM checkpoint achieves 99.40% contract‑valid coverage versus 37.83% for a string‑first baseline while maintaining balanced predictive performance. Additional evaluations on SciTS, TSQA, IRTS‑ToolBench, and ARFBench demonstrate the model’s transferability.

By Jiahui Chen, Bingke Zhu, Hongyu Pan, Yingying Chen
arXiv Machine Learning
Sep 17

A GAN-Based Framework for Robust DDoS Attack Detection

The paper introduces a GAN‑based framework for detecting DDoS attacks that are designed to evade traditional security systems. It combines Random Forests, Deep Neural Ensembles, and Transformer models trained on the CICDDoS2019 dataset with synthetic adversarial traffic generated by a WGAN‑GP. Experiments show that this hybrid training significantly improves detection accuracy and resilience against unseen adversarial traffic, and real‑world tests confirm its practical effectiveness.

By Makram Chehayeb, Walid Fahs, Amina Rizk, Rida Khatoun, Omran Berjawi