← Back to all news
arXiv AI By Catherine King, Lynnette Hui Xian Ng, Kathleen M. Carley

Beyond Effectiveness: A Multi-Criteria Framework for Comparing Practical Socio-Technical Interventions

Read the original on arXiv AI →

The paper introduces a multi‑criteria framework for evaluating sociotechnical interventions, moving beyond sole effectiveness to include political feasibility, user acceptance, cost, and implementation effort. Using misinformation as a case study, the authors surveyed 39 researchers on 40 interventions across these five criteria, revealing that the most effective solutions are not always the most acceptable or feasible. The study offers a decision framework to help practitioners balance these trade‑offs in designing interventions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 9

How Is Automated Research Evaluated? A Survey of Benchmarks and Evaluation Practices

The paper surveys how automated research systems—used for literature synthesis, ideation, workflows, writing, peer review, and end‑to‑end research—are evaluated. It compares task construction, evidence sources, evaluators, and scoring methods across six target areas, revealing that output checks, process checks, and human studies together provide a fuller picture. The authors identify gaps in diagnostic designs and offer reporting and audit recommendations to guide future evaluation efforts.

By Liulei Zhang, Dejing Zhou, Chuyue Huang, Guanhua Chen, Yutong Yao, Lidia S. Chao, Chi Man Vong, Derek F. Wong
benchmarks
More like this →
arXiv AI
Jul 1

RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studies

arXiv:2603. 11001v3 Announce Type: replace-cross Abstract: Human uplift studies, or studies that measure the effects of AI access on human performance via randomized controlled trials (RCT) or similar methodologies, increasingly inform frontier AI governance and deployment decisions.

By Patricia Paskov, Kevin Wei, Shen Zhou Hong, Dan Bateyko, Xavier Roberts-Gaal, Carson Ezell, Gailius Praninskas, Valerie Chen, Umang Bhatt, Ella Guest
llms
More like this →
arXiv Machine Learning
Jun 4

Validity Threats for Foundation Model Research

arXiv:2606. 05029v1 Announce Type: new Abstract: Controlled experiments are the backbone of machine learning research, but at the scale of modern foundation models, they have become prohibitively expensive.

By Gunnar K\"onig, Martin Pawelczyk, Ulrike von Luxburg, Sebastian Bordt
More like this →