← Back to all news
arXiv Machine Learning By Aleesha Zainab, Muhammad Ahmed Khalid, Faheem Ullah Khan, Asifullah Khan

Benchmarking Classical and Transformer-Based Models for Document Sensitivity Classification

Read the original on arXiv Machine Learning →

The paper introduces Strategic 16K, a 16,000‑document corpus of diplomatic cables from WikiLeaks’ Public Library of US Diplomacy, designed to eliminate label leakage. It benchmarks six models—both classical machine learning and transformer-based—on this leakage‑controlled dataset, finding BERT and ELECTRA top performers while TF‑IDF with Logistic Regression offers strong accuracy at lower cost. This work provides the first fully reproducible sensitivity‑classification benchmark built under explicit leakage‑control conditions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 23

What Does 99% Accuracy Measure? A Reproducible Audit of Shortcut Learning in a Widely Used Fake News Corpus

The paper audits the widely used ISOT/Kaggle Fake and Real News corpus and finds that extremely high reported accuracies (≈0.98) are largely due to shortcut signals rather than genuine veracity detection. A simple TF‑IDF linear classifier achieves perfect F1 when using only subject metadata, and even after removing metadata, newswire tags, and duplicate documents, the F1 drops only modestly, indicating that editorial style rather than specific tokens drives performance. Under topic‑disjoint and temporal transfer tests, performance collapses, and models transfer poorly to the independent LIAR benchmark, showing that within‑corpus scores reflect source and topic separability, not truth verification. whyItMatters:"The study demonstrates that current high accuracy metrics on this fake‑news dataset are misleading, highlighting the need for more robust evaluation protocols that guard against shortcut learning."

By Yuvraj Verma
fine-tuningbenchmarks
More like this →
arXiv AI
Aug 3

TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Text

arXiv:2607. 28862v1 Announce Type: cross Abstract: The rapid development of Large Language Models (LLMs) has led to significant advances across a wide range of language tasks, while simultaneously raising growing concerns about unauthorized data exploitation and privacy leakage.

By Chengshuai Zhao, Pingchuan Ma, Dawei Li, Bohan Jiang, Zhiyuan Yu, Zhen Tan, Huan Liu
llmsnlpreinforcement-learningfine-tuningbenchmarkssafety
More like this →