← Back to all news
arXiv Machine Learning By Xianling Zhang

C-Instrument: Automating RL Data Generation and Hillclimbing with a Constitution-Grid Instrument

Read the original on arXiv Machine Learning →

The paper introduces C-Instrument, a constitution‑grid data instrument designed to automate reinforcement learning (RL) data generation and hillclimbing. It also presents C‑LIM, a per‑cell learnability score that guides actions such as prune, densify, amend, or expand, and identifies dead‑weight data regions before training. The authors report that their method improves over‑refusal rates from 22.4% to 12.8% and reduces under‑refusal on adversarial attacks from 0.27 to 0.33, while lifting learning impact in a previously ineffective data region from 0.733 to 0.80.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 8

Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks

The paper examines the vulnerability of open‑weight large language model safeguards that aim to block harmful usage by assuming new harmful behavior arises only through fine‑tuning. It demonstrates that simple, low‑cost attacks—abliteration and prefilling—can significantly increase attack success rates across multiple benchmarks, and proposes an abliteration‑resistant tuning (ART) method that mitigates these attacks. The study shows that safeguarding defenses must be evaluated against a broader range of attack strategies beyond fine‑tuning.

By Kevin Kuo, Virginia Smith, Chhavi Yadav
llmsfine-tuningbenchmarkssafety
More like this →
arXiv Computation and Language
Oct 19

Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data

The paper introduces EDIC, a framework for per‑instruction compliance in large language models that interact with external data. It decomposes the task into identifying, classifying, and acting on each embedded directive, using a scene‑grounded pipeline and a generative judge trained with GRPO to attribute compliance at the instruction level. Experiments on two backbones and five benchmarks show that EDIC improves safety and utility, achieving up to a 7.7% increase in harmful‑rejection and a 12.5% boost in Arena‑Hard‑Auto utility on Qwen3‑4B.

By Zesheng Shi, Xue Li, Chenrui Dai, Jing Li
llmsreinforcement-learningfine-tuningbenchmarkssafety
More like this →