Outdated numeric values from Norwegian statutes and regulations in a further-trained model and its base model: a preregistered paired comparison of Borealis 2 and Gemma 4

The study compares two Norwegian language models—Borealis 2, further‑trained from Gemma 4, and Gemma 4 itself—on how often they provide outdated numeric values from Norwegian statutes and regulations. Using the closed norregel benchmark, the models were evaluated on 503 primary items; Borealis 2 was outdated on 10 items and Gemma 4 on 11, a difference that is not statistically significant and falls within the pre‑registered smallest effect of interest. The only secondary measure that differed after Holm correction was the share of answers giving a value that took effect after the question date, with Gemma 4 giving such answers more often.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• benchmarks