AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,798 stories · RSS feed

arXiv Computation and Language
Sep 22

When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

arXiv:2609.22102v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts...

By Anubhav Gupta, Mayara Costa Figueiredo, Leticia Santos Machado, Tanner Wright, Ivan Beschastnikh, Cleidson R. B. de Souza, Gema Rodr\'iguez-P\'erez
arXiv Computation and Language
Sep 22

SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine

arXiv:2410.17021v2 Announce Type: replace Abstract: Large Language Models with chain-of-thought prompting, such as OpenAI-o1, have shown impressive capabilities in natural language inference tasks. H...

By Xiaochen Wang, Liang Chen, Reza Haf Zhe Yang, Yiru Wang, Xiangdi Meng, Kunhao Pan, Zhifang Sui, Junqing He
arXiv Computation and Language
Sep 22

ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost?

arXiv:2605.00468v2 Announce Type: replace Abstract: Plain Language Summaries (PLS) aim to make research accessible to lay readers, but they are typically written in a one-size-fits-all style that ign...

By Joey Chan, Yikun Han, Jingyuan Chen, Samuel Fang, Lauren D. Gryboski, Alexandra Lee, Sheel Tanna, Qingqing Zhu, Zhiyong Lu, Lucy Lu Wang, Yue Guo
arXiv Computer Vision
Sep 22

Document Retrieval-Aware Chunking (D-RAC): Universal Retrieval-Aware Ingestion of Enterprise Documents via PDF Normalization and Multimodal Markdown Conversion

arXiv:2609.24220v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presen...

By Uday Allu (AI Research Team Yellow.ai), Abhivanth Sivaprakash (AI Research Team Yellow.ai), Pratik Singh (AI Research Team Yellow.ai), Aman Manocha (AI Research Team Yellow.ai)
arXiv Machine Learning
Sep 22

Computationally efficient safe exploration in reinforcement learning

The paper introduces “CoLSafe-MDP”, a reinforcement learning algorithm that ensures safe exploration in constrained Markov decision processes. It replaces computationally heavy Gaussian process methods with a Nadaraya-Watson estimator, achieving constant-time scaling for estimate bounds. The authors evaluate the algorithm on a grid-based environment and on observational Martian terrain data.

By Shreeram Murali, Shankar A. Deka, Dominik Baumann