arXiv AI By Ting Gong, Shitan Xu

TwistedMerge: Certified Higher-Order Diagnostics and Abstention for Model Merging

Read the original on arXiv AI →

arXiv:2607. 20887v1 Announce Type: cross Abstract: Model merging combines independently trained or fine-tuned models, but pairwise alignability does not imply globally consistent alignment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko
arXiv Machine Learning
Oct 8

When Rank Rises as LLMs Degrade

The paper investigates how spectral statistics, particularly RankMe and covariance effective rank, behave when large language models (LLMs) undergo post‑training in non‑stationary environments. In controlled experiments on Qwen3‑0.6B, data duplication causes a 75% increase in held‑out loss while simultaneously raising both original and centred RankMe, indicating spectral dispersion rather than rank collapse. The authors demonstrate that a two‑sided, multichannel sequential monitor can detect various damage regimes but does not predict held‑out loss earlier and requires healthy calibration data.

By Zhaohui Geoffrey Wang