Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters

The paper investigates how open‑book scoring evaluates document‑level LoRA adapters by training one adapter per Wikipedia article on Llama‑3.1‑8B‑Instruct. Results show that open‑book scores largely reflect a shared reading behaviour: a random adapter can recover most of a specialist’s advantage when the document is provided, but only a negligible fraction when the document is omitted in a cloze probe. While specialists improve cloze recall and their weight updates align with the documents they read, selecting adapters by weight‑space proximity or text embedding does not outperform random choice, and merged adapters recover most of the open‑book advantage but only a small portion of the cloze advantage. The study also documents four silent errors that yielded clean but incorrect results.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• rag
• fine-tuning