← Back to all news

AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computer Vision
Sep 22

Semi-automated reconstruction of indoor geometry from 360-degree video for CFD-based airflow analysis in classrooms

The paper presents a semi‑automated pipeline that transforms a single 360‑degree video of a classroom into editable, simulation‑ready 3D geometry for CFD analysis. Using NeRF for dense point clouds, SAM‑based 2D instance masks lifted to 3D, and octree‑based instance separation, the workflow produces object‑level assets that can be reconfigured in a browser editor. The resulting geometry is validated with OpenFOAM simulations and benchmarked against IEA Annex 20, demonstrating that per‑room geometry acquisition is essential for accurate airflow predictions.

By Dhruv Gamdha, James Afful, Shambhavi Joshi, Ulrike Passe, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
computer-visionbenchmarkssafetydata-engineering
More like this →
arXiv Machine Learning
Sep 22

Toward Fairness in Machine Learning Models for Predicting Treatment Retention and Premature Discontinuation in Medication for Opioid Use Disorder

The paper evaluates machine learning models that predict retention and premature discontinuation in medication for opioid use disorder (MOUD). Using the Treatment Episode Data Set-Discharges (TEDS‑D) from 2015‑2019, the authors trained four models and examined overall performance as well as subgroup error rates by race, ethnicity, age, and sex. They also tested bias‑mitigation techniques, finding that these can reduce but not eliminate performance gaps without harming predictive accuracy.

By Tongnian Wang, Carolina Vivas-Valencia, Cici Bauer, Yanmin Gong, Kim-Kwang Raymond Choo, Yuanxiong Guo
safety
More like this →
arXiv Machine Learning
Sep 22

Robust Market Making with Hawkes Order Flow and Price Impact via Adversarial Reinforcement Learning

The paper proposes an adversarial reinforcement‑learning framework for market making that incorporates Hawkes‑process driven order arrivals and trade‑induced price impact, addressing limitations of prior Poisson‑based models. An LSTM module captures temporal dependencies in recent observations to handle increased non‑stationarity, and the authors analyze equilibrium properties and introduce a robustness evaluation protocol focused on the left tail of returns. Experiments across diverse market regimes demonstrate that the method improves left‑tail performance, especially under strong Hawkes excitation and moderate price impact, without relying on a terminal inventory bias.

By Hao Yang, Zhenguo Xu
reinforcement-learningsafety
More like this →
arXiv Machine Learning
Sep 22

A Pinch of SFT, A Dash of RL: When Reinforcement Learning Helps Long-Horizon Advertising Agents

The paper investigates how to balance supervised fine‑tuning (SFT) and reinforcement learning (RL) for long‑horizon advertising agents that must retrieve, reason, and execute code on distributed business data. By classifying training trajectories into Imitation, Lift, and Discovery regimes, the authors develop a diagnostic that predicts which training path to follow, and then apply a targeted SFT‑then‑RL strategy. Across 18 feature‑specific experiments on GPT‑OSS 120B, this approach yields positive gains on most advertiser skills, reduces leakage, and achieves higher mean improvements with 43% less RL compute.

By Aakash Kolekar, Sahika Genc, Bunyamin Sisman, Shahriar Shariat, Shree Vandana Kachroo, Avishek Saha, Qianli Wu, Ari Singer, Benoit Dumoulin
llmsagentsreinforcement-learningfine-tuningsafety
More like this →
arXiv Machine Learning
Sep 22

EvoRank: LLM-Guided Evolution of Multi-Objective Learning-to-Rank Pipelines

EvoRank is an open autonomous ranking engineer that uses an LLM-guided evolutionary loop to automatically design complete Learning-to-Rank pipelines—including features, models, losses, and ensembles—for multi-objective e-commerce search. On the Expedia ICDM 2013 dataset, EvoRank converged within 50 iterations on interpretable pipelines that outperform an Optuna-tuned LambdaMART on 60k held-out queries and rank in the top 6 % of the original competition. The authors also introduce a headroom gate that predicts whether the evolutionary loop will be worthwhile before any LLM computation, and they release the system, auditing tools, and a catalog of failure modes to help teams apply the method to their own ranking stacks.

By Rayhan Patel, Shabaz Patel
llmsagentssafety
More like this →
arXiv Machine Learning
Sep 22

Predictors and Orchestrators: Parsimonious Machine Learning within an Agentic AI Harness for Multi-Horizon Karst Aquifer Forecasting

The study presents a deployment‑aware framework for forecasting spring discharge and groundwater levels in the Edwards Aquifer over 1‑12 week horizons using 79 years of hydroclimatic data. Five machine‑learning families—extreme gradient boosting, extremely randomized trees, LSTM, CNN, and Transformers—were compared, with extreme gradient boosting consistently delivering the highest reliability (R² ≥ 0.94) and strong agreement with operational drought thresholds. The validated models were integrated into a five‑agent operational architecture that automates data acquisition, model selection, prediction, threshold monitoring, verification, literature retrieval, and reporting.

By Pramod Lekhak, Chetan Sharma, Hakan Ba\c{s}a\u{g}ao\u{g}lu, F. Paul Bertetti, Debaditya Chakraborty
llmsagentssafety
More like this →
arXiv Machine Learning
Sep 22

Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation

Teacher Should Think Ahead: Adaptive Continuations for Reliable On-Policy Distillation explores how on‑policy distillation (OPD) can be improved by addressing teacher uncertainty that contracts as the teacher continues from a student‑generated prefix. The authors identify Teacher Uncertainty Contraction (TUC) and theoretically analyze its variance‑bias trade‑off, leading to the proposal of Adaptive‑Continuations On‑Policy Distillation (AC‑OPD). Experiments on mathematical reasoning and code generation show that AC‑OPD consistently outperforms standard OPD, with controlled‑continuation and matched‑budget analyses supporting the adaptive‑continuation design.

By Jingang Zhou, Yuyi Zhou, Haiyang Guo, Xukai Wang, Shuai Feng, Sirui Gao, Jian Xu, Qingpei Guo, Xu-Yao Zhang
llmsefficiencysafety
More like this →
arXiv Computation and Language
Sep 22

Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling

arXiv:2609.22934v1 Announce Type: new Abstract: Large language models (LLMs) increasingly mediate human decisions and communication, yet their behavioural regularities remain difficult to characteriz...

By Yu Sha, Junqi Tao, Dixin Zhou, Yansheng Tu, Mingyang Chen, Xiang Fan, Yang Liu, Mengquan Yang, Jie Lin, Jiahui Fu, Hua Zheng, Benwei Zhang, Zhou Kai
llmssafety
More like this →