Alignment, interpretability, red-teaming, bias and privacy: the research on what these systems do when they misbehave.
We study learning a discrete distribution under person-level differential privacy in the batched setting. Given $n$ users, each contributing $m$ i.
The paper introduces continuous diffusion language models (dLLMs) to replace the random mapping used in continuous-space black-box attacks on large language models. By sampling or optimizing directly in a learned generative latent space, dLLMs produce more natural prompts and significantly reduce character-level perplexity in both sampling-based (RareTrap) and optimization-based (ThinkTrap) evaluations. The study also shows that risk estimates can vary by an order of magnitude depending on whether a random mapping or a learned diffusion model is used.
IndoNutriQA is a curated benchmark of 911 Indonesian clinical nutrition question‑answer pairs derived from real reports. Each pair includes a clinically relevant context, structured metadata, and answers in free‑form prose, with expert validation indicating 6‑10% of answers are contestable. The dataset supports generative, extractive, and retrieval‑augmented QA, and is used to evaluate fine‑tuned models and large language models in zero‑ and few‑shot settings.
The paper investigates why long image captions often fail, arguing that failure depends not just on what facts are mentioned but also on where they appear in the caption. It introduces a position‑aware rubric that breaks down dense human references into atomic visual claims, evaluates generated captions against these claims, and identifies two distinct failure modes: unsupported specificity in the middle and vague, non‑evidential language at the end. Building on this diagnosis, the authors propose PAGER, a reinforcement‑learning objective that provides separate rewards for supported middle evidence, evidence‑bearing tail continuation, claim coverage, and length control, leading to improved position‑wise grounding and caption preference on detailed‑caption benchmarks.
Lean Finder V2 is a semantic search framework for Mathlib that tackles the problem of short, ambiguous user queries by using adversarial training and query rewriting to disambiguate them. It achieves a new state‑of‑the‑art performance on the MathlibQR benchmark, improving retrieval by at least 11% over previous engines, and further boosts proof success when integrated with Claude Code.
The paper introduces TESSA, a governance-oriented Custom GPT designed to aid small and medium-sized enterprises (SMEs) in complying with the European Union Artificial Intelligence Act (EU AI Act). Using a design science research approach, TESSA was evaluated against baseline ChatGPT across 100 benchmark scenarios, showing higher governance alignment rates (90% vs. 83%) and consistent performance in repeated executions. The study presents a human-validated regulatory update mechanism and a multidimensional evaluation framework for governance-oriented AI artifacts.
Training large language models for complex reasoning tasks relies on two complementary signals: sparse, outcome-grounded reinforcement learning (RL) for correctness, and dense, token-level distillatio...
Cancer care is a longitudinal, time-sensitive process requiring coordination among diverse healthcare professionals (HCPs), whose collaboration evolves throughout treatment. Prior studies link care te...
The paper introduces ShadowBench, a benchmark built from real photographs that tests vision‑language models (VLMs) on three shadow‑related tasks: reading a visible shadow’s direction, inferring where a shadow should fall when absent, and judging physical consistency of shadows. Evaluation of 19 open VLMs and proprietary models shows that while they can read shadow direction well, they perform poorly on inferring absent shadows and judging consistency, with only Gemini‑3.1‑pro slightly above chance. The authors demonstrate that routing a VLM through an image‑generation model (IGM) provides the missing knowledge, significantly improving inference and highlighting the potential of image‑generation objectives for illumination‑aware VLMs.
The study evaluates VideoGen Model 1, a post‑trained version of MiniMax H3, by comparing raw, reformatted, and expanded prompts across 30 audiovisual generation tasks. Creative prompt expansion yielded higher overall quality scores than formatting alone, with a 0.63‑point increase under VA‑Judger and a 0.15‑point increase under Gemini 3.8 Flash, while prompt‑alignment scores rose by 0.97 and 0.50 points respectively. Audio quality improved notably with VA‑Judger, whereas visual quality remained largely unchanged.
The study evaluates whether ‘abliterated’ (uncensored) language models truly eliminate political bias by comparing matched pairs of 2‑bit models—NVIDIA’s Nemotron 3.5 Bolt 74B‑A9B and Qwen3.8‑Flash‑Next—against community‑created abliterations. Across Traditional Chinese, Simplified Chinese, and English, the models’ responses about Taiwan’s status varied more with script than with abliteration; even after removing refusals, the models still affirmed that Taiwan is part of the PRC, especially in Simplified Chinese. The research shows that removing refusal mechanisms does not eradicate script‑dependent bias, and that the choice of script can significantly influence a model’s political stance.
StructLens is a framework that analyzes language model representations by constructing maximum spanning trees from semantic representations in residual streams, inspired by dependency parsing. It reveals that contiguous tokens are close in representation space, with middle layers showing the strongest local-span organization. The study also shows that smaller local units emerge earlier during pre‑training, while larger units appear later, offering insights into how token representations are organized across layers and training stages.
The paper introduces conditional imaging, an approach that alters the optical state for each captured frame so that the measurement alone hides the scene while reconstruction requires knowledge of the system state. The authors prove that static optical systems cannot simultaneously guarantee privacy and recoverability, and they design a camera prototype that demonstrates this trade‑off on a human‑face dataset. Experiments show that authorized users can reconstruct identity, whereas observers lacking the system state cannot retrieve visual information.
Think-Capture is a query‑conditioned framework that selectively acquires 3D geometry for task‑critical objects to improve spatial reasoning in Vision‑Language Models. It uses a 3D toolkit to lift zero‑shot masks into a common coordinate frame, a Symbolic 3D State Abstraction to encode object and camera states as structured text, and Hierarchical Group Relative Policy Optimization to train the VLM on when geometry helps or misleads. The resulting Think‑Capture‑4B model achieves significant gains on MindCube‑Tiny, MMSI‑Bench, and ViewSpatial benchmarks while remaining robust to perturbations and fitting on a single 24 GB GPU.
The paper introduces Role-Enhanced Zero-Shot Chain-of-Thought (RE‑ZS‑CoT), a prompting framework that incorporates domain‑specific expert roles to improve reasoning in large language models without task‑specific examples. Two variants are evaluated: a static version that tests eight predefined roles across arithmetic, commonsense, and reading comprehension tasks, and a dynamic version where the model selects the most suitable role for each benchmark. Across 20 LLMs and seven benchmarks, RE‑ZS‑CoT consistently outperforms zero‑shot and zero‑shot‑CoT baselines, achieving up to 44 percentage‑point gains and demonstrating strong alignment between static and dynamic approaches, while ablation studies confirm the importance of task‑role alignment.
The paper reviews and categorizes large language models that combine step‑by‑step reasoning with interactive search, treating retrieval as an action within an iterative agentic loop. It analyzes these systems along four architectural axes—retrieval coupling, controller design, knowledge source, and grounding strictness—while also considering the learning signal as a cross‑cutting factor. The authors conduct controlled ablations and benchmark analyses to clarify how design choices affect search, stopping, and grounding behavior, aiming to provide a unified perspective for future research.
SANE is an agent‑based framework that automatically generates scorer‑specific perturbation recipes and expected score relations for large‑language‑model judges, using behavioral requirements encoded as reusable agent instructions. It validates each recipe against the scorer’s intent and the instantiated test case, producing a growing inventory of aligned recipes as more scorers are added. In experiments across five domains, SANE yields 3.7× more validated test families than the Judge Reliability Harness, detects deficiencies in all domains—including professionalism and toxicity—and achieves strong human alignment in judge ranking on faithfulness (Spearman ρ = 0.852).
MINER is a model-agnostic framework designed to improve clothes-changing person re-identification by suppressing garment bias. It works by contrasting hard examples and adaptively balancing intra- and inter-person variance in the dataset. Experiments demonstrate that MINER achieves state-of-the-art performance on four CC-ReID benchmarks and ranks among the leaders in all three tracks of the open TVRID RGB-Depth ReID challenge.
The paper introduces a trace-level divergence framework for evaluating multilingual large language models, breaking down agreement into structural similarity, order consistency, and holistic similarity. Experiments across three reasoning benchmarks, seven languages, and six leading LLMs show that conventional holistic similarity metrics overstate agreement, with up to 67.3% of top-scoring examples lacking structural alignment. Human judgments align more closely with structural similarity, highlighting language-specific divergence and the need for trace-level evaluation in multilingual deployments.
CanLegalRAGBench is a Canadian legal QA benchmark that uses realistic queries and expert‑annotated answers grounded in case law to evaluate retrieval‑augmented generation (RAG) systems. The study finds that retrieval performance varies with design choices and that open‑source embedding models can compete with closed‑source ones. It also highlights limitations of automatic evaluations, noting that RAG systems often produce hallucinations or overly detailed responses, with 8‑29% of claims unsupported by retrieved documents.