arXiv Machine Learning September 25, 2026 By Wenjun Qiu, David Lie, Lisa Austin

Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning

Read the original on arXiv Machine Learning →

Calpric is a system that combines automatic text selection, segmentation, active learning, and crowdsourced annotation to create a large, balanced training set for privacy policy classification. By simplifying the labeling task, it enables untrained crowd workers to match the performance of trained annotators and reduces inter‑annotator disagreement, cutting labeling costs. The approach yields a dataset of 16,000 policy text segments across nine data categories and produces models that deliver accurate, fine‑grained labels at a cost of roughly $0.92–$1.71 per segment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 22

Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning

arXiv:2401.08038v2 Announce Type: replace-cross Abstract: A significant challenge to training accurate deep learning models on privacy policies is the cost and difficulty of obtaining a large and com...

By Wenjun Qiu, David Lie, Lisa Austin

computer-vision safety

arXiv AI

Jun 9

Deep Active Re-Labeling: Toward Noise-Resilient Annotation Efficiency

arXiv:2606. 08718v1 Announce Type: cross Abstract: While Deep Active Learning (DAL) effectively reduces human annotation costs, its efficacy is constrained by human annotation errors.

By Md Abdullah Al Forhad, Weishi Shi

arXiv Machine Learning

Jul 7

Labeled-Data-Free Meta-Learning: Efficient Task Generation Using Pre-trained Models and Unlabeled Data

arXiv:2607. 02850v1 Announce Type: new Abstract: Meta-learning without labeled data is crucial for real-world applications, where obtaining labeled datasets can be expensive or restricted due to privacy concerns.

By Lei Sun, Yusuke Tanaka, Tomoharu Iwata

benchmarks safety

arXiv Machine Learning

Jul 31

When unlearning is free: leveraging low influence points to reduce computational costs

arXiv:2512. 05254v2 Announce Type: replace Abstract: As concerns around data privacy in machine learning grow, the ability to unlearn, or remove, specific data points from trained models becomes increasingly important.

By Anat Kleiman, Robert Fisher, Ben Deaner, Udi Wieder

safety

arXiv Machine Learning

Jul 21

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

arXiv:2310. 16152v5 Announce Type: replace-cross Abstract: Federated learning (FL) has become a key component in various language modeling applications such as machine translation, next-word prediction, and medical record analysis.

By Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, Shagufta Mehnaz

llms nlp safety

arXiv AI

Jun 17

Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection

arXiv:2604. 13899v3 Announce Type: replace-cross Abstract: Instruction-tuned LLMs can annotate thousands of instances at low cost.

By Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein, Hinrich Sch\"utze

llms

One email a morning, machine-written

Related stories

Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning

Deep Active Re-Labeling: Toward Noise-Resilient Annotation Efficiency

Labeled-Data-Free Meta-Learning: Efficient Task Generation Using Pre-trained Models and Unlabeled Data

When unlearning is free: leveraging low influence points to reduce computational costs

Gradient-Free Privacy Leakage in Federated Language Models through Selective Weight Tampering

Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection