QuasiLeak-Bench: Evaluating LLM-Based Privacy Detectors on Policy-Conditioned Quasi-Evidence Group Recovery

QuasiLeak-Bench is a new benchmark that evaluates large language model (LLM) based privacy detectors on their ability to recover quasi‑evidence groups—sets of weak quasi‑identifiers that jointly reveal private attributes—while respecting sender‑specified disclosure policies. The benchmark uses a reverse‑construction pipeline to generate documents containing verified quasi‑evidence groups and applies automated quality control to produce reliable ground truth at scale. Experiments show that current privacy detectors perform poorly on this task, but a fine‑tuned 4B LLM can outperform the best prompted models, indicating the task is learnable.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• fine-tuning
• benchmarks
• safety