arXiv AI By Shubhadip Nag, Srinjoy Das, Agniva Saha, Anushree Ghosh, Soumi Das, Tarun Kumar, Suparna Bhattacharya, Sourangshu Bhattacharya

MPSelectTune: Prompt-type Selection for Fine-tuning improves Concept Unlearning in LLMs

Read the original on arXiv AI →

arXiv:2607. 03932v1 Announce Type: cross Abstract: LLMs can be conveniently adapted to a diverse set of tasks, e.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Aug 28

$p1$: Better Prompt Optimization with Fewer Prompts

The paper investigates why prompt optimization works better for some tasks than others by decomposing reward variance into response variance and system‑prompt variance. It finds that optimization succeeds when system‑prompt variance dominates, and that adding more user prompts can actually reduce this variance, especially on heterogeneous datasets. To address this, the authors propose $p1$, a filtering method that selects a small set of high‑variance user prompts, which improves optimization on reasoning benchmarks and even allows a system prompt trained on just two AIME 24 prompts to generalize well.

By Zhaolin Gao (Sid), Yu (Sid), Wang, Bo Liu, Thorsten Joachims, Kiant\'e Brantley, Wen Sun
arXiv Computation and Language
Sep 1

HiVe: Beyond Static Prompts for Multitask Learning via Hierarchy-based Vertical Mixture-of-Experts

HiVe is a prompt‑tuning framework that builds a hierarchy of prompts by exploiting inter‑task relationships during training. It uses a vertical mixture‑of‑experts (V‑MoE) at inference to compose prompts at the level of specialization needed for each input, allowing input‑dependent prompt adaptation. Experiments demonstrate that HiVe consistently outperforms strong prompt‑tuning baselines across diverse tasks.

By HyeonJik Bae, Minyeol Kim, Susik Yoon