The study evaluates whether ‘abliterated’ (uncensored) language models truly eliminate political bias by comparing matched pairs of 2‑bit models—NVIDIA’s Nemotron 3.5 Bolt 74B‑A9B and Qwen3.8‑Flash‑Next—against community‑created abliterations. Across Traditional Chinese, Simplified Chinese, and English, the models’ responses about Taiwan’s status varied more with script than with abliteration; even after removing refusals, the models still affirmed that Taiwan is part of the PRC, especially in Simplified Chinese. The research shows that removing refusal mechanisms does not eradicate script‑dependent bias, and that the choice of script can significantly influence a model’s political stance.
arXiv:2608. 11816v1 Announce Type: cross Abstract: State-aligned distortion has been documented in China-origin text-based large language models (LLMs), but whether, and in what form, it arises in multimodal systems has not been systematically examined.
arXiv:2610.19848v1 Announce Type: new Abstract: We present OpenTWBench, an open evaluation suite for Taiwan-domain knowledge in large language models, built around two benchmarks. tw-exam-bench conta...
arXiv:2603.18280v4 Announce Type: replace-cross Abstract: Current alignment evaluation mostly measures whether models encode dangerous concepts and whether they refuse harmful requests. Both miss the...
arXiv:2611.03171v1 Announce Type: cross Abstract: On July 1, 2025, Vietnam reduced its 63 provinces to 34, abolished its 696 districts, and replaced its 10,035 commune-level units (wards) with 3,321....
The paper introduces a method for measuring political positions in data‑sparse regions by treating a large language model as a single fallible rater within a panel, analogous to expert surveys. It demonstrates that this panel approach yields reliable agreement across multiple models and reveals that disagreements often stem from differing interpretations of the regime rather than factual errors. The authors provide a full rubric, prompt, and data set, using the Middle East and North Africa as a worked example.
The paper "Benchmarking LLM Compliance with China AI Generated Content Regulations" presents a new framework for evaluating large language models (LLMs) against China’s AI‑generated content compliance rules. Using 2,303 questions across six dimensions—including 203 self‑constructed constitutional queries—the authors assess 20 prominent LLMs, finding that international models show high compliance, especially on ideologically aligned dimensions. The study establishes a regulatory benchmark that allows the global AI community to test both Chinese and non‑Chinese LLMs under unified, legally grounded compliance criteria.
BanglaBBQ is a question‑answering bias benchmark that captures Bangladeshi stereotypes in both Bangla and English. It adapts the original BBQ by classifying templates into directly translated, modified for Bangladesh, and removed categories, retaining 140 templates across six groups. Five new stereotype categories—marital status, age and political perception, regional origin, educational background, and mental‑health status—were added, with 466 new templates evaluated and 372 retained after annotation. The dataset was tested on Qwen3‑4B, Qwen3‑8B, GPT‑4.1‑mini, and Gemini 2.5 Flash, revealing that all models tend to favor stereotyped individuals when context is ambiguous, with language‑specific differences and varying accuracy depending on stereotype confirmation or contradiction.
The paper investigates how large language models (LLMs) represent nine transborder ethnic communities along China’s southwestern border with mainland Southeast Asia, such as the Sami, Rohingya, and Kurds. It introduces the concept of identity ossification, where LLMs reduce fluid, cross-border identities to a single nation-state narrative, and presents the Transborder Identity Ossification Diagnostic Framework (TIODF) to differentiate between knowledge gaps and narrative framing. Experiments with GPT‑5.1 and DeepSeek‑V3.2 in English and Chinese show that ossification rates vary widely, with DeepSeek‑V3.2’s Chinese responses exhibiting the phenomenon in 62.6% of cases, indicating that narrative framing rather than mere knowledge absence drives the issue.
arXiv:2605. 25420v2 Announce Type: replace-cross Abstract: Large language model safety evaluation remains heavily English-centered, leaving low-resource languages under-measured even when models are deployed globally.
arXiv:2608.21821v1 Announce Type: new Abstract: When Wikipedia's language editions describe the same concept, how differently do they frame it? Prior work measures coverage gaps between editions; we...
PowerBench is a new evaluation framework that measures how language models respond to power‑shifting requests, distinguishing self‑empowerment, disempowerment, and power grabbing while controlling for neutral requests. The authors built and released a dataset covering different power domains, contexts, scales, and prior power standings, and tested 24 models from US and Chinese developers across three experimental conditions. Results show models are more likely to refuse power‑grabbing than disempowerment, and that biases exist toward helping US users gain power while resisting US users taking power from others, with additional effects when the user is an AI agent or when requests are in different languages.
arXiv:2606. 18699v1 Announce Type: cross Abstract: Large language models (LLMs) have shown impressive capabilities across diverse tasks, yet their performance on jurisdiction-specific legal reasoning remains underexplored.