AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,788 stories · RSS feed

arXiv Machine Learning
5d ago

A Unified Matrix-Spectral Framework for Stability and Interpretability in Deep Learning

The paper introduces a unified matrix‑spectral framework that treats deep neural networks as data‑dependent products of linear operators, enabling the analysis of stability and interpretability through spectral quantities. It defines a Global Matrix Stability Index that aggregates spectral information from Jacobians, parameter gradients, Neural Tangent Kernel operators, and loss Hessians into a single stability scale, and shows that spectral entropy provides a more realistic measure of sensitivity than worst‑case bounds. Experiments on MNIST, CIFAR‑10, and CIFAR‑100 demonstrate that modest spectral regularization can significantly improve attribution stability without large changes to global spectral summaries.

By Ronald Katende
arXiv Computation and Language
5d ago

Do Open Language Models Invent Scripture? BILAL-BENCH: Benchmarking Islamic-Source Fabrication in Bengali

The paper introduces BILAL-BENCH, a 240-question benchmark designed to evaluate open-ended Islamic-source fabrication in Bengali. It assesses models on Qur'anic attribution, hadith, fiqh, contemporary issues, and adversarial prompts, using a rubric that separates source errors from assertiveness. Seven open-weight models were tested, revealing that most confidently fabricate on 79.3–95.3% of source-production questions, with hadith provenance being the weakest area and no clear improvement with larger model size.

By Sabah Nushra, Md Kishor Morol
arXiv Computation and Language
5d ago

Service, Solidarity, and Self-Help: A Comparative Topic Modeling Analysis of Community Unionism in the Boot and Shoe Union and Unite Community

This paper compares community unionism in the National Boot and Shoe Union of the 1920s with Unite Community of the 2010s–2020s using BERTopic and cTF-IDF. It finds that Unite Community’s discourse aligns more with outward‑facing, social‑justice themes, whereas the Boot and Shoe Union focuses on internal administration, industrial relations, and member services. The study demonstrates how modern NLP can illuminate differences in historical labor archives and challenges assumptions about the continuity of community unionism.

By Thomas Compton
arXiv AI
5d ago

Cross-Interface Authorization for AI Computer-Use Agents

The paper introduces a protocol for cross‑interface authorization of AI computer‑use agents, ensuring that policy decisions remain consistent across graphical interfaces, command lines, APIs, scripts, and Model Context Protocol servers. It defines a profile‑bounded system that enumerates all effect‑bearing routes, normalizes requests to canonical effects, issues prepare tickets, and re‑verifies all relevant state at commit time, requiring a sink receipt for a committed result. Extensive testing across two runtime mappings produced thousands of traces, with rigorous checks confirming no semantic regressions, mutant rejections, and correct outcome matching.

By Genliang Zhu, Chu Wang
arXiv AI
5d ago

Whose Values, Which Contexts, and What Signals? Grounding Cultural Value Alignment with Re3Culture

arXiv:2610.41793v1 Announce Type: new Abstract: Cultural value alignment requires not only model optimization but also a well-specified target grounded in real-human evidence. Yet how such targets sh...

By Jing Yao, Hyunjin Hwang, Weihua Zheng, Bryan Chen Zhengyu Tan, Jia Wang Peh, Dylan Raharja, JinYeong Bak, Roy Ka-Wei Lee, Rui Cao, Yuki Arase, Jacki O'Neill, Zhicheng Dou, Xing Xie, Xiaoyuan Yi, Furu Wei
arXiv AI
5d ago

P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior

P-ContextBench is a benchmark designed to evaluate how well large language models understand and safely use personal context. It creates realistic, life‑story‑grounded digital humans from U.S. demographic seeds, embedding traits, routines, goals, and constraints in fragmented, evolving synthetic mobile‑app records. The benchmark tests five capabilities—fact recovery, temporal tracking, pattern inference, and safe handling of conflicting requests—using a multi‑stage validation pipeline and 4,810 verified test cases.

By Grant Nasir Brown, Abdul Basir Bari, Anmol Tadikonda, Isabel Song
arXiv AI
Oct 31

Proving and Improving the Platonic Representation Hypothesis with JEPAs

The paper proves the Platonic Representation Hypothesis by showing that Joint-Embedding Predictive Architectures (JEPAs) trained on different modalities converge to the same representation up to a rotation when their positive pairs are good views of the same scene. It demonstrates that next‑token prediction is a JEPA with bad views, explaining why language models only align with vision through large scale. Experiments on millions of captions and web images confirm a scaling law of alignment, revealing that good views require far fewer samples than next‑token prediction and that a 109M‑parameter text JEPA outperforms larger language models in aligning with image encoders, even enabling a simple rotation to produce an image captioner with high COCO CIDEr scores.

By Randall Balestriero, Yann LeCun
arXiv AI
Oct 31

From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe

The paper examines how post‑training changes in large language models affect fact‑checking judgments based on a poster’s social class. Using randomized poster profiles, the study finds that only the open T"ulu‑3 recipe introduces a statistically reliable bias—making the model more likely to label claims from lower‑status jobs as false—while other commercial models do not exhibit this effect. The bias is traced to a single residual‑stream direction that emerges during supervised fine‑tuning, and the authors confirm the mechanism on additional claims.

By Nouar AlDahoul, Yasir Zaki
arXiv AI
Oct 31

Auditable Belief Updates as a Missing Safety Layer for LLM-Powered Multi-Agent Systems

The paper proposes the Explicit Belief Multi-Agent System (EB‑MAS) to address safety in large language model (LLM) powered multi‑agent systems. It treats belief as an engineering abstraction for evidence‑based commitments and externalizes belief states into a structured substrate, using belief operators to propose updates and a Cognitive Firewall to filter weakly evidenced changes. The authors also highlight risks such as manufactured consensus, collusion, and cascading misinformation, and discuss practical challenges in grounding, false positives, scalability, and ethical misuse.

By Zhuoyun Li, Boxuan Wang, Jinwei Hu, Xiaowei Huang, Yi Dong
arXiv AI
Oct 31

Beyond LLM Wrappers: Context Engineering and the Three-Engine Architecture for Reliable Financial AI

The paper proposes a three‑engine architecture for reliable financial AI, combining a financial knowledge graph, a deterministic optimization engine, and a multi‑agent LLM layer. A coordination layer called context engineering selects and validates query‑specific context from ERP systems and other tools. Observations from 15 deployments show that multi‑agent validation reduces hallucination rates to below 1%, while reusable mappings and templates cut deployment time from six to two weeks.

By Pradeep Kumar GS, Ravi Tanniru