← Back to all news
arXiv Machine Learning By Zhaohui Geoffrey Wang

When Rank Rises as LLMs Degrade

Read the original on arXiv Machine Learning →

The paper investigates how spectral statistics, particularly RankMe and covariance effective rank, behave when large language models (LLMs) undergo post‑training in non‑stationary environments. In controlled experiments on Qwen3‑0.6B, data duplication causes a 75% increase in held‑out loss while simultaneously raising both original and centred RankMe, indicating spectral dispersion rather than rank collapse. The authors demonstrate that a two‑sided, multichannel sequential monitor can detect various damage regimes but does not predict held‑out loss earlier and requires healthy calibration data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Sep 4

Contamination Inflates Scores but Rarely Reorders Large Language Model Leaderboards

The paper investigates how benchmark contamination—leakage of test items into training data—affects large language model (LLM) leaderboards. By comparing original test items with semantically equivalent paraphrases, the authors measure contamination as a violation of anchor-item invariance and find that it inflates absolute scores but rarely changes model rankings. Across 47 public models and 74 finetuned models on four benchmarks, the rank correlation between standard and paraphrase-controlled leaderboards is 0.997, with only a handful of cases showing differential contamination that could alter rankings.

By Xingyao Xiao (Stanford University), Yihong Cheng (City University of Macau)
llmsbenchmarks
More like this →