Hugging Face Trending Papers
CompOrca: Corpus-Scale Compliance Labelling of Instruction-Tuning Data
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
2d ago
CompOrca: Corpus-Scale Compliance Labelling of Instruction-Tuning Data
arXiv:2609.37807v1 Announce Type: cross Abstract: Studying how fine-tuning shapes refusal and noncompliance behaviour requires identifying training examples that refuse, evade or otherwise fail to fu...
Hugging Face Trending Papers
Jun 24
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring
Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade. The judge is rarely checked.
arXiv Machine Learning
Jun 25
How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring
arXiv:2606. 25487v1 Announce Type: cross Abstract: Almost every paper on LLM jailbreaks and prompt injection reports an attack-success rate (ASR), and that number is assigned not by people but by an automated judge: either a safety classifier trained for the task, or a general chat model prompted to grade.
arXiv Computation and Language
Sep 22
Used, Mentioned, or Condemned? A Controlled Contrast-Set Diagnostic for the Use-Mention Distinction in Code-Mixed Hinglish Misogyny Detection
The paper introduces a diagnostic tool for distinguishing the use of misogynistic slurs from their mention in counter‑speech within code‑mixed Hinglish. It identifies evaluation artifacts in existing corpora, releases a 416‑item minimal‑pair contrast set that decorrelates slur presence and gendered register from labels, and proposes a pair‑consistency metric to assess model performance. Experiments show that even strong baselines struggle to consistently label counter‑speech pairs, while a large language model achieves perfect scores, indicating the benchmark measures genuine capability rather than exploitation of artifacts.
arXiv AI
Aug 3
The Asymmetric Effects of Knowledge Distillation on Bias in Small Language Models
arXiv:2607. 28639v1 Announce Type: cross Abstract: We show that knowledge distillation in small instruction-tuned language models has asymmetric effects on bias.
arXiv Computation and Language
Sep 22
The Copy Ceiling: An Input-Exposure Control for Ontology-Grounded Generation over Curated Corpora
arXiv:2609.24885v1 Announce Type: new Abstract: When a language model answers from a curated corpus via graph-based retrieval, a large grounding uplift does not establish reasoning over the retrieved...