AI safety and alignment

Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.

17,808 stories · RSS feed

arXiv Computer Vision
Sep 11

HeteroPROMPT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework

HeteroPROMPT is a real‑time, privacy‑preserving framework for heterogeneous collaborative perception in autonomous systems. It aligns features from diverse sensors and models into a unified ego‑centric space using modular prompts and lightweight tuning, while keeping encoders and fusion stacks frozen. The system employs a metadata‑free autoencoder for modality classification and routing, achieving higher average precision on OPV2V‑H and V2XSet datasets with far fewer trainable parameters.

By Armin Maleki, Hayder Radha
arXiv Computer Vision
Sep 11

RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition

RAIDAL is an active learning framework for continuous sign language recognition that leverages the CTC decoder’s alignment peaks to focus sample selection on gloss‑aligned regions, thereby avoiding temporal redundancy in weakly aligned videos. By restricting representation‑based scoring to these decoder‑aligned gloss areas, RAIDAL improves data efficiency across multiple datasets and architectures, especially in large‑vocabulary, budget‑limited scenarios. The method requires no extra labeling cost and its implementation is publicly available on GitHub.

By Rafael A. Diniz Augusto, Gabriel L. Oliveira, Erickson R. Nascimento
arXiv AI
Sep 11

Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations

The paper examines whether internal representations of agentic systems can better indicate task success than traditional confidence measures. It introduces two methods—Latent Trajectory Dynamics (LTD) and Action Representation Probe (ARP)—that analyze changes in residual-stream representations and action-level representations, respectively. Experiments on Bash, SQL, and Python benchmarks with Qwen and DeepSeek models show these methods outperform conventional surface-level and sequence-based calibration baselines, offering a zero‑overhead reliability monitor without prompt changes or multiple rollouts.

By Priyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla
arXiv AI
Sep 11

Procedural Memory Under Change: Reuse and Interference in Controlled Web Tasks

The paper investigates how procedural memory behaves when its stored routines become mismatched with changing interfaces. Using a BrowserGym TimeWarp case and controlled synthetic shopping tasks, the authors found that procedural memory can be mismatched without causing behavioral disruption, as none of the predefined interference signatures appeared. The study highlights a specific region of non‑interference but leaves open which conditions would turn mismatches into observable errors.

By Yanze Cao
arXiv AI
Sep 11

AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents

AgentAudit is an open, extensible framework that evaluates the full lifecycle of AI agents, assessing planning, tool selection, execution, memory, and reasoning across ten dimensions such as instruction integrity, security, and alignment. Unlike existing benchmarks that focus on single aspects, AgentAudit analyzes the entire execution trace to attribute failures to specific stages. The framework was tested on five large language models, revealing significant differences in trustworthiness even among models with similar task‑completion performance.

By Shrey Nag, Sachita, Abhishek Kumar Singh, Lipi Goel, Rajeshwar Singh Janwar
arXiv Computer Vision
Sep 11

HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA

HALDETECT is a system developed for the English hallucination-detection track of ImageEval 2026, where the task is to identify the single visually grounded statement among three culturally plausible options. The approach treats the problem as a contrastive decision, outputs the answer before an explanation, and bases reasoning on colour/texture, shape/form, and context. The best model fine‑tunes Qwen2.5‑VL‑7B‑Instruct with 4‑bit QLoRA, freezes the vision encoder, and achieves a Contrastive Instability score of 0.035 on the test set, placing third among eight teams.

By Syed Mohaiminul Hoque, Md Sakhawat Hossain
arXiv Computer Vision
Sep 11

Language-Augmented Semantic Priors for B-Spline Surface Fitting

The paper introduces LASP, a framework that uses large language models to generate structured B‑spline priors from procedural modeling histories. By translating design intent into rich textual descriptions, LASP provides semantic reasoning that guides conventional CAD solvers toward more accurate and coherent surface fitting. Experiments show that language‑driven priors outperform traditional machine learning approaches, establishing a new paradigm for language‑guided geometric optimization.

By Yunzhong Lou, Yusheng Luo, Jiahao Li, Yu Song, Xiangdong Zhou
arXiv Machine Learning
Sep 11

SG-Blend: Learning an Interpolation Between Improved Swish and GELU for Robust Neural Representations

SG-Blend introduces a per‑layer adaptive activation that interpolates between a bias‑corrected, parametric Swish variant (SSwish) and GELU, using a learnable blend coefficient, sharpness, and zero‑centering bias. The method adds only three scalars per feed‑forward block and, on BERT‑style IMDB classification, matches peak accuracy while reducing seed‑to‑seed variance by 42 %. It also achieves the lowest validation perplexity on WikiText103 and generalizes to computer vision and other domains.

By Gaurav Sarkar, Syed Affan Daimi, Jay Gala, Subarna Tripathi
arXiv Machine Learning
Sep 11

Evidence for Limited Metacognition in LLMs

The paper introduces a new method for measuring metacognitive abilities in large language models (LLMs) without relying on self-reports, instead testing how well models can use knowledge of their internal states. Using two experimental paradigms, the authors find that recent frontier LLMs can assess and use their own confidence when answering factual and reasoning questions, and can anticipate and appropriately employ the answers they would give. The study also shows that these abilities are limited in resolution, context-dependent, differ qualitatively from human metacognition, and vary across models with similar capabilities, suggesting post‑training processes influence metacognitive development.

By Christopher Ackerman
arXiv Computation and Language
Sep 11

Probing for Knowledge Attribution in Large Language Models

The paper introduces a method for identifying the dominant knowledge source behind large language model (LLM) outputs, distinguishing between faithfulness violations (misuse of provided context) and factuality violations (errors in internal knowledge). A simple linear probe trained on hidden representations can reliably classify this source, and the authors present AttriWiki, a self‑supervised pipeline that generates labeled training data by prompting models to recall withheld entities or read them from context. Probes trained on AttriWiki achieve high Macro‑F1 scores across several models and datasets, generalize zero‑shot to a benchmark, and show that attribution mismatches can increase error rates by up to 70%. "whyItMatters":"The study demonstrates that knowing the source of an LLM’s answer is crucial for effective mitigation of hallucinations, as attribution mismatches significantly raise error rates."

By Ivo Brink, Alexander Boer, Dennis Ulmer
arXiv Computer Vision
Sep 11

Domain Elastic Transform: Bayesian Function Registration for High-Dimensional Scientific Data

Domain Elastic Transform (DET) is a grid‑free, probabilistic framework that jointly aligns geometry and high‑dimensional vector‑valued functions on irregular sparse manifolds, such as those found in spatial transcriptomics. By treating data as functions rather than voxelized images, DET performs unsupervised, scalable registration through sampled point alignment and displacement interpolation, guided by a joint spatial‑functional Bayesian likelihood. Evaluations on MERFISH mouse‑brain slices and Stereo‑seq mouse‑embryo atlases show DET achieving superior spatial overlap and topology compared to existing pipelines, with an accelerated variant delivering high label‑transfer accuracy.

By Osamu Hirose, Emanuele Rodola
arXiv Machine Learning
Sep 11

AUC Maximization from Biased Positive-unlabeled Data with Confidence

The paper introduces a method for maximizing the area under the receiver operating characteristic curve (AUC) when only biased positive and unlabeled (PU) data are available. It leverages confidence scores—probabilities that an instance is positive—associated with a small set of labeled positives to derive an AUC risk estimator that accounts for bias. Experiments on eight real-world datasets demonstrate the method’s effectiveness.

By Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara
arXiv Machine Learning
Sep 11

LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation

LOCUS is a post‑training technique that selects a task‑aware low‑rank adaptation subspace to reduce token generation cost while preserving a model’s preference alignment. By updating only a tiny fraction of parameters (0.24–0.28 %) on 3 B‑parameter backbones, LOCUS cuts continuation length by up to 39.84 % on Pythia‑2.8B and 14.87–17.58 % on Qwen2.5‑3B, without materially affecting internal preference diagnostics. The method demonstrates that careful subspace selection can make large language models more token‑efficient without sacrificing utility.

By Dongfang Zhao
arXiv Machine Learning
Sep 11

Domain-Specific Hallucination Detection in Large Language Models

The paper introduces a multi‑signal pipeline for detecting hallucinations in large language models, combining fine‑tuned DeBERTa‑v3 classification, Monte Carlo Dropout uncertainty, and temperature‑scaled calibration. On the HaluEval benchmark it achieves high performance (F1 = 0.915, AUROC = 0.977) across QA, summarization, and dialogue, and shows that 25 % of training data yields 77 % of full‑data performance. The authors also demonstrate that applying Direct Preference Optimization to a Qwen2.5‑0.5B generator cuts hallucination rates from 85.5 % to 37.7 %, and that domain‑specific fine‑tuning (PubMedBERT on SciFact) outperforms general‑domain models for biomedical text.

By Varun Teja Chundru, Debasmita Biswas
arXiv Computation and Language
Sep 11

Beyond Prompting: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integration (LOGIC)

The paper introduces LOGIC (Logit‑Space Integration for Contextual Biasing), a new framework that injects contextual entity information directly into the decoding layer of Speech Large Language Models, bypassing the limitations of prompt‑based methods. LOGIC operates with constant‑time complexity regardless of the size of the entity list, and experiments with the Phi‑4‑MM model across 11 multilingual locales show an average 9% relative reduction in Entity WER while adding only a 0.30% increase in False Alarm Rate.

By Peidong Wang, Jian Xue, Jinyu Li
arXiv Computation and Language
Sep 11

Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding

The paper introduces IRS (Incongruity-Resolution Supervision), a framework that breaks humor understanding into three parts: identifying mismatches in a visual scene, creating coherent reinterpretations of those mismatches, and aligning these interpretations with human preferences. IRS uses structured reasoning traces to guide models from visual perception to humorous interpretation, and it is evaluated on the New Yorker Cartoon Caption Contest. Experiments on 7B, 32B, and 72B models show that IRS improves caption matching and ranking, with the 72B model achieving 76.10% ranking accuracy—outperforming non-expert humans and all other multimodal baselines—and demonstrates transferable reasoning patterns in zero‑shot settings.

By Hatice Merve Vural, Doga Kukul, Ege Erdem Ozlu, Demir Ekin Arikan, Bob Mankoff, Erkut Erdem, Aykut Erdem
arXiv Machine Learning
Sep 11

Federated Learning for Surgical Vision in Appendicitis Classification: Results of the FedSurg EndoVis 2024 Challenge

The FedSurg Challenge is the first international effort to evaluate Federated Learning (FL) for surgical vision, using a multi‑center dataset of laparoscopic appendectomies. Three participant models were tested for generalization to an unseen clinical center and for center‑specific adaptation, compared against centralized, Swarm Learning, and parameter‑efficient fine‑tuning baselines. The study found that temporal modeling most consistently improves generalization, but overall performance remains low (26.31% F1‑score on the unseen center), highlighting the need for structured personalized FL and revealing limitations of current approaches.

By Max Kirchner, Hanna Hoffmann, Alexander C. Jenke, Oliver L. Saldanha, Kevin Pfeiffer, Weam Kanjo, Julia Alekseenko, Claas de Boer, Santhi Raj Kolamuri, Lorenzo Mazza, Nicolas Padoy, Sophia Bano, Annika Reinke, Lena Maier-Hein, Danail Stoyanov, Jakob N. Kather, Fiona R. Kolbinger, Sebastian Bodenstedt, Stefanie Speidel
arXiv Computation and Language
Sep 11

A Training-Free, Alignment-Free Approach to Corporate Intelligence: Application to SEC Filings

The paper introduces a training‑free, alignment‑free method for corporate intelligence that uses deterministic sparse seed vectors to hash word strings into a fixed high‑dimensional basis. By accumulating these seed vectors across sentence contexts, the authors create corpus‑specific semantic signatures that enable rapid document comparison, issuer fingerprinting, vocabulary shift tracking, and thematic sentence extraction—all on standard CPU hardware. Applied to a multi‑year set of SEC filings, the approach reveals distinct semantic profiles for major corporate events such as Boeing’s 737 MAX crisis, Intel’s supply‑chain disruptions, and Bunge’s acquisition of Viterra, with each profile traceable to its source sentences without any domain‑specific training or LLM inference.

By Jean-Fran\c{c}ois Delpech
arXiv Machine Learning
Sep 11

HuRo: Robotizing Human Videos for Scalable VLA Pretraining

The paper introduces HuRo, a dataset of 630K robotized episodes derived from diverse human videos, created via a pipeline that aligns observations and actions for robotic use. Experiments on four real‑world manipulation tasks show that scaling robotized pretraining boosts task completion from 51.5% to 80.3% and improves out‑of‑distribution performance under spatial and visual shifts. Ablation studies reveal that visual robotization enhances robustness and that end‑to‑end pretraining with retargeted actions outperforms visual‑only transfer.

By Jinho Jeong, Se June Joo, Jaehyun Kang, Dongyun Kim, Yena Kim, Hanjung Kim, Seon Joo Kim
arXiv Computer Vision
Sep 11

AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flow

AcFlow introduces an inference‑time controller for text‑to‑image diffusion transformers that transports intermediate layer activations through a learned, concept‑conditioned velocity field while keeping the base model frozen. The method allows fine‑grained style intensity control and suppression of unwanted concepts, achieving superior style–content trade‑offs compared to baselines and generalizing to unseen concepts without per‑concept fitting. Experiments demonstrate improved style alignment and qualitative suppression of diverse concepts where direct prompting fails.

By Junran Wang, Zehao Jin, Tianyu Luan, Xinjie Shen