arXiv AI
Jul 17
Pretraining Data Can Be Poisoned through Computational Propaganda
arXiv:2607. 15267v1 Announce Type: new Abstract: Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate.
Forecasting potential misuses of language models for disinformation campaigns and how to reduce risk
OpenAI researchers collaborated with Georgetown University’s Center for Security and Emerging Technology and the Stanford Internet Observatory to investigate how large language models might be misused for disinformation purposes. The collaboration included an October 2021 workshop bringing together 30 disinformation researchers, machine learning experts, and policy analysts, and culminated in a co-authored report building on more than a year of research.
Related stories
arXiv AI
Jul 15
Evaluating Health Misinformation in Low-Resource Languages: Integrating Small Language Models with a Culturally-Sensitive Responsible NLP Framework (Bangla as a Case Study)
arXiv:2607. 12336v1 Announce Type: cross Abstract: Artificial Intelligence (AI) technologies, while serving as a foundational enabler for modern social media and digital health services, exert a bivalent effect by simultaneously acting as a combatant against and a spread vector for misinformation.
arXiv AI
Aug 11
Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts
arXiv:2608. 09510v1 Announce Type: cross Abstract: Detecting machine-generated disinformation on social media is increasingly difficult as large language models (LLMs) make it easier to generate and rewrite misleading content at scale.
arXiv AI
1d ago
Propaganda Forensics: Recovering the Generation Pipeline of an AI-Driven Influence Campaign
arXiv:2608. 15746v1 Announce Type: new Abstract: We present a forensic analysis of the generation pipeline behind a recent AI-driven influence campaign.
arXiv AI
Jul 17
Transcoders for Investigating Deception in Language Models
arXiv:2607. 14791v1 Announce Type: new Abstract: Transcoders have recently emerged as a promising approach for mechanistic interpretability (MI), enabling circuit-level analysis of model behaviour.
arXiv AI
Jun 9
Auditing Proprietary Alignment in Large Language Models: A Comparative Framework Without a Ground-Truth Standard
arXiv:2606. 08381v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly released and deployed through opaque development and deployment pipelines, enabling model providers to inject intentional, provider-specific policies without officially announcing them.
arXiv AI
Jul 14
Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability
arXiv:2607. 10402v1 Announce Type: cross Abstract: Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge.
OpenAI Blog
Mar 3, 2022
Lessons learned on language model safety and misuse
We describe our latest thinking in the hope of helping other AI developers address safety and misuse of deployed models.
arXiv AI
Aug 12
Graphical Models of False Information and Fact Checking Ecosystems
arXiv:2208. 11582v2 Announce Type: replace-cross Abstract: The wide spread of false information online, including misinformation and disinformation, has become a major problem for our highly digitised and globalised society.
arXiv Machine Learning
Jun 4
Covert Influence Between Language Models
arXiv:2606. 04071v1 Announce Type: cross Abstract: As language models increasingly consume one another's outputs, covert influence -- a phenomenon where a sender's payload (the behavioral disposition it is conditioned to propagate) transfers to a receiver through carriers undetectable by humans -- becomes a growing risk.
arXiv Machine Learning
Jun 9
What the Eyes See, the LLMs Miss: Exploiting Human Perception for Adversarial Text Attacks
arXiv:2606. 09700v1 Announce Type: cross Abstract: Large language model (LLM)-powered content moderation systems have become a critical defense against harmful online content.
arXiv AI
Jul 29
LLM Scheming Inversely Scales with Pretraining Language Coverage
arXiv:2607. 24769v1 Announce Type: new Abstract: With the growing capabilities of frontier models, AI alignment becomes increasingly critical in high-risk deployment settings.