Form Over Content In Gradient-Based Data Attribution Methods

The paper investigates what gradient similarity measures in data attribution for large language models. By independently varying task and answer format in supervised fine‑tuning benchmarks, the authors show that gradient alignment is driven by answer format rather than task semantics, with strong alignment for shared formats and none for differing formats. This pattern persists across training stages, model sizes, and families, and is evident in the selections of the LESS data‑selection method, which over‑represents its own answer format.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• fine-tuning
• benchmarks
• safety