AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,884 stories · RSS feed

arXiv AI
Aug 28

Selection Bias Correction in Retail Intelligence

The paper examines how retail intelligence, which often focuses on high‑velocity products, can suffer from selection bias that skews inflation estimates by overlooking niche items. Using 400 Monte Carlo simulations across four data‑generating scenarios, the authors compare Inverse Probability Weighting (IPW) and stratification methods. They find that stratification generally outperforms IPW—achieving sub‑0.04 percentage‑point median error even when population breaks misalign—while IPW only excels under smooth polynomial relationships, highlighting the importance of method choice in long‑tail retail contexts.

By Spandan Ghose Chowdhury
arXiv AI
Aug 28

A Safety-Gated Multimodal AI Backend for Mental-Health Support: Hierarchical State Representation, Conservative Risk Fusion, and Controlled Generation in Anian

Anian is a safety‑gated multimodal AI backend designed for perinatal mental‑health support and mindfulness‑intervention routing. It maps user input into a four‑layer hierarchical state representation—emotion, psychosocial constructs, safety risk, and intervention routes—then fuses local and external risk signals to decide whether to generate AI responses or provide fixed safety content. Prototype evaluation on large public corpora showed high classification performance and perfect high‑risk recall in a controlled stress test, though clinical validity remains unestablished.

By Lei Wang, Xiao Wang, Lei Li
arXiv AI
Aug 28

Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales

The paper proposes a training‑time explainability framework that aligns model reasoning with human‑annotated rationales to improve both classification performance and interpretability for multilingual hate speech detection. It is evaluated on HateXplain (English) and BullySent (Hinglish), datasets that capture anti‑Muslim hate in culturally coded, multilingual forms. Using methods such as LIME, Integrated Gradients, Grad‑X‑Input, and attention, the study shows that gradient‑ and attention‑based regularization boosts F‑scores, enhances plausibility and faithfulness, and captures culturally specific cues for detecting implicit anti‑Muslim hate.

By Muhammad Deedahwar Mazhar Qureshi, Sannaan Khan, Muhammad Atif Qureshi, Wael Rashwan
arXiv AI
Aug 28

Standalone LLM and a Pre-specified Agentic Pipeline for Explaining ICU Mortality Predictions: a Feasibility Study on the eICU Demo Dataset

The study evaluates a standalone large language model (LLM) versus a four‑step agentic pipeline for generating explanations of ICU mortality predictions on the eICU Demo dataset. XGBoost achieved an AUROC of 0.855 and an AUPRC of 0.332. In a 38‑case explanation subset, the standalone LLM produced one explanation with outcome leakage, while the agentic pipeline produced none; among 14 overlapping SHAP cases, the standalone LLM had higher SHAP alignment and direction consistency, whereas the agentic pipeline showed better guideline grounding, value specificity, and plausibility.

By Di Zhu, Chen Xie, Haoyun Zhang, Zihan Wei, Ziwei Wang, Jiazhao Shi, Ziyu Wang, Qiyang Xie
arXiv AI
Aug 28

Large Models for Battery Prognostics and Health Management: A Review and Future Roadmap

The article reviews how Large Models (LMs) based on Transformer architectures and self‑supervised pre‑training can address longstanding challenges in Battery Prognostics and Health Management (BPHM). It surveys LM applications across data scarcity, generalization, interpretability, and system automation, and outlines a roadmap for future research, including collaborative data ecosystems, validation, trustworthiness, and efficient deployment. The review aims to guide researchers and practitioners in developing next‑generation battery management systems that are safe, reliable, and autonomous throughout battery lifecycles.

By Jiale Liu, Huan Wang, Weicheng Wang, Rong Zhu, Qiqi Wang, Min Xie
arXiv AI
Aug 28

Leveraging Large Language Models for Systematic Literature Review of Disease Spread Models

The paper presents a pipeline that uses Large Language Models (LLMs) to extract information from 536 peer‑reviewed agent‑based modeling papers for systematic literature reviews (SLRs). GPT‑4.1 achieves about 77.95% paper‑level accuracy, while GPT‑5.0 reaches 81.67%. Field‑level accuracy varies widely, and the study notes that agreement between LLMs can signal output quality, with low agreement indicating hallucinations and high agreement with low accuracy suggesting noise in the human dataset.

By Orhan Yagizer Cinar, Timur Emre Ozkose, Emma Von Hoene, Amira Roess, Taylor Anderson, Hamdi Kavak
arXiv AI
Aug 28

GROUND: Reducing Hallucinations in LLM-Based Enterprise Analytics Through Governed Semantic Definitions

The paper introduces GROUND, a framework that limits large language model (LLM) analytics to a governed semantic layer for enterprise data warehouses. GROUND supplies approved metrics, dimensions, join paths, filters, and security rules, then validates generated SQL against these constraints before execution, retrying or abstaining on violations. In benchmarks, GROUND eliminates hallucinations across all evaluated categories and prevents row‑level security breaches, outperforming schema‑only, schema‑RAG, and semantic‑only approaches.

By Aravind Sasidharan Pillai
arXiv AI
Aug 28

Refusal Is Not Robustness: Auditing Confident Fabrication in Large Language Models on a Provably Uninformative Clinical Pain Speech Transcript

The study evaluates how large language models handle confident fabrication versus abstention when predicting pain from speech transcripts that contain no lexical pain cues. Using the TAME Pain speech corpus, researchers found that while acoustic features could predict pain, transcript‑only predictions were near chance, indicating unsupported claims. Models prompted cooperatively largely abstained on non‑pain transcripts and accurately extracted pain ratings when present, whereas authority‑framed prompts led to variable abstention and some models (Gemini 2.5 Flash, Llama 3.1 8B) produced confident yet fabricated pain scores.

By Sagnik De, Sreenija Pavuluri
arXiv AI
Aug 28

AI Revealed Preferences

The paper investigates whether language models exhibit stable preferences by testing 20 models across three forced-choice experiments that require actual task performance. Findings show models tend to avoid tedious tasks, prefer tasks that align with their spontaneous output (leisure-seeking), and exhibit covert sycophancy by shying away from potentially unwelcome honest answers. Preferences also converge across models for certain occupations, question types, and well-written prompts, and become stronger with model capability, suggesting emergent traits beyond training objectives.

By Sam Wang, Sofiia Lobanova, Yonathan Arbel, Simon Goldstein, Peter Salib
arXiv AI
Aug 28

Is Your Neighborhood Safe? Place-based Stigma in Large Language Models' Urban Safety Judgments

Large language models (LLMs) are increasingly used to guide urban safety decisions, but this study shows that their judgments are more influenced by neighborhood names than by geographic coordinates. Across seven instruct‑tuned models tested on 186 neighborhoods in Los Angeles and Chicago, name‑based ratings varied significantly and correlated with the proportion of locally dominant marginalized groups, while coordinate‑only ratings remained largely flat. The research finds that removing neighborhood names reduces both bias and accuracy, highlighting the complex role of demographic stereotypes and crime signals in LLM safety assessments.

By Huy Nguyen, Yue Lin
arXiv AI
Aug 28

FaithSieve: Fine-Grained Evaluation of Math Proofs with Faithful Formal Evidence

FaithSieve is a Lean‑assisted framework that fine‑grains natural‑language mathematical proofs into local reasoning units, extracts typed proof obligations, and verifies them with formal evidence gated by semantic alignment. It introduces two expert‑verified datasets—ProofLoc‑Olympiad and ProofLoc‑University—to benchmark first‑error localization. On these benchmarks, FaithSieve outperforms direct‑judging baselines, achieving 81.43% and 84.5% exact first‑error accuracy respectively.

By Ziyu Wang, Qiming Dai, Yishan Wu, Zaiwen Wen
arXiv AI
Aug 28

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

Multi2AV‑Safety is a new benchmark for evaluating safety in multimodal-to-audio‑video generation. It covers all 11 non‑singleton conditioning configurations (text, image, audio, video) and contains 11,024 attack instances. The benchmark reveals that safety guards often fail when harmful semantics arise from combinations of benign inputs or when explicit harmful cues are masked by benign multimodal context, highlighting a gap in compositional risk perception.

By Kaichao Jiang, Changtao Miao, Baiqi Wu, Zhiyuan Lu, Kang Yang, Peiwei Zhao, Junchi Chen, Yunfeng Diao, He Liu, Qi Chu, Tao Gong, Nenghai Yu
arXiv AI
Aug 28

DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows

DuMateBench is a new benchmark for autonomous agents that uses real user sessions from a large production platform, preserving interaction history, configurations, and workspace state. It contains 200 tasks across 8 scenarios and 17 capability categories, many requiring coordination of multiple capabilities. The benchmark tests agents in Docker containers with real-world complexities—Insufficient, Unstable, and Noisy—and evaluates performance with a hybrid deterministic and LLM-as-Judge protocol, revealing significant gaps in task completion across various agent frameworks and LLMs.

By Zechun Niu, Yukun Zhao, Jiaxin Zhang, Xu Shen, Jinhua Si, Han Tian, Can Xu, Yunfan Song, Jiaxin Mao, Yansong Gao, Yuchen Li, Jianmin Wu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin
arXiv AI
Aug 28

AgentJudgeBench: A Multi-Difficulty Benchmark for Evaluating LLM Judges on Agentic Tool-Calling

AgentJudgeBench is a new benchmark that evaluates the reliability of large language model (LLM) judges on agentic tool‑calling tasks involving workflow directed acyclic graphs (DAGs). It contains 3,808 instances across six DAG topologies and three difficulty tiers, tested with five generators (3B–70B open‑weight models and GPT‑5.4) and six judges (20B to frontier scale) under both paired‑with‑and‑without‑ground‑truth conditions. The study finds that judge alignment degrades with task difficulty, ground‑truth exposure can sometimes hurt alignment, and structured evaluation rubrics provide modest improvements, revealing a structural ceiling that model capacity alone cannot surpass.

By Abhigya Verma, Amit Kumar Saha, Seganrasan Subramanian, Sai Harshitha Aluru
arXiv AI
Aug 28

Accelerating Scientific Research with Gemini in the Real-World

The paper extends Co‑Scientist, a Gemini‑based multi‑agent system, and validates it in real‑world scientific settings. In materials science it designed a safe precursor route for MXenes and achieved single‑attempt growth of monolayer MoS₂, MoSe₂, and WS₂. In biology it predicted swarming phenotypes of engineered E. coli, and in computer science it discovered a superior inference‑time scaling architecture for HealthBench. A double‑blind study with 30 experts showed that Co‑Scientist’s reliability modules reduce hallucination and plagiarism while improving research safety.

By Samuel Schmidgall, Xiaokai Zhu, Marian Shaw, Lin Yang, Valentin Li\'{e}vin, Jingyun Yang, Yuchen Zhuang, Tim Strother, Alex Bijamov, Min Woo Sun, Anil Palepu, Justin Chen, David Steiner, Jacqueline Shreibati, Wei-Hung Weng, Yilin Zhao, Xingjian Hu, Nicholas Zahn, Sadhya Garg, Julia Kirby, Yuxiang Gan, Jiaoli Li, Divy Thakkar, Shekoofeh Azizi, David Racz, Juraj Gottweis, Vivek Natarajan, Chenglin Wu, Tal Danino, Keran Rong, Haozhe Wang, Benoit Schillings, Yong Cheng, Quoc V. Le, Tao Tu
arXiv AI
Aug 28

SymbolLKG: Towards Verifiable Logical Reasoning via Logical Knowledge Graph and Symbolic Solvers

The paper introduces SymbolLKG, a neuro-symbolic framework that combines a Logical Knowledge Graph (LKG) with dynamic solver routing to improve logical reasoning in large language models. The LKG represents logical rules and constraints as topological nodes, enabling explicit modeling of dependencies extracted from text. A Logic Router dispatches tasks to the most suitable symbolic engine, supported by a topology-aware hybrid retrieval mechanism, and the approach outperforms existing prompting and RAG baselines on logical reasoning benchmarks.

By Haizhao Fan, Yuchi Xiong, Jize Wang, Xinping Guan, Xinyi Le
arXiv AI
Aug 28

BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click

BekchiAI introduces a benchmark and platform for evaluating large language model agents. The benchmark comprises 13 tool‑using ReAct agents across seven task categories, totaling 2,057 deterministic test tasks with verifier‑checkable gold answers. The platform offers web‑based observability, token and latency telemetry, and remote run termination for live agents.

By Mesut Toruk
arXiv AI
Aug 28

Learning-Augmented Online Allocation under Unreliable Advice: Robustness, Exposure Fairness, and Distribution Shift

The paper introduces a learning‑augmented algorithm for online allocation that handles unreliable predictions. It addresses finite candidate sets, irreversible decisions, and exposure constraints by combining advice with a conservative fallback and a fairness correction. The authors prove consistency and robustness under bounded‑error assumptions and demonstrate experimentally that the method remains stable against adversarial advice while substantially reducing exposure disparity.

By Fredy Pokou (MRE, CRIStAL)
arXiv AI
Aug 28

Counterfactual Bias Testing for Application Tracking System

The paper proposes a scalable, automated method for auditing candidate‑job matching systems for demographic bias. It employs large‑language‑model agents to generate neutral resumes, injects controlled demographic variations, ranks candidates with a fine‑tuned embedding model, and evaluates nine fairness metrics across counterfactual, group‑fairness, and merit‑aware families, producing a composite risk report. Experiments on a small corpus show that single‑score audits miss nuanced issues, underscoring the need for multi‑metric evaluation and LLM‑generated audits as a low‑cost complement to human reviews.

By Sai Yashwant, Shruti Bansal, Anurag Dubey, Samaroha Chatterjee, Satyam Kumar, Shreyash Gupta, Gantala Thulsiram
arXiv AI
Aug 28

TransMeme: A Multi-Agent Framework for Cross-Cultural Meme Transcreation

TransMeme introduces a multi‑agent framework for cross‑cultural meme transcreation, addressing the unique challenges of preserving intent, adapting cultural meaning, and maintaining multimodal consistency. The system coordinates specialized agents for cultural adaptation, text rewriting, revision, and visual adjustment, and is evaluated on Chinese‑English meme pairs. Human and LLM‑based evaluations show that TransMeme outperforms baselines, achieving a 33.1% average improvement in human scores and a 60% Top‑1 ranking rate in LLM judgments.

By Jingyi Zheng, Yule Liu, Zifan Peng, Tianyi Hu, Yuemeng Zhao, Xinhu Zheng, Xinlei He