← Back to all news
arXiv Machine Learning By Haaris Mian, Miles Cranmer, Alessandro Favero, Payel Mukhopadhyay

Frozen in Time: How a Physics Foundation Model Forgets Under Sim-to-Real Adaptation

Read the original on arXiv Machine Learning →

The paper investigates catastrophic forgetting in the scientific foundation model Walrus-1.3B during a two‑stage sim‑to‑real adaptation. Finetuning first on simulated flow around a cylinder quickly degrades performance on the original pretraining data, with one‑step errors rising up to 35× after 20 epochs, and further adaptation to experimental data worsens this degradation. The authors show that replaying as little as 1% of the simulated data during real‑data finetuning largely restores simulation performance while still adapting effectively to real dynamics.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 22

Combating Catastrophic Forgetting in Continual Domain Adaptation via Knowledge Recasting

The paper introduces Skill Schema Transport (SST), a post‑training framework that mitigates catastrophic forgetting in continual domain adaptation by translating new domain knowledge into a model’s existing skill space. SST abstracts each example type into a reusable skill schema, preserving essential operations while removing domain‑specific surface forms, thereby turning knowledge updates into procedural reuse rather than trajectory memorization. Experiments on two 3B instruction‑tuned backbones show that SST consistently reduces forgetting and improves generalization, achieving up to a 5.75‑point performance gain and a dramatic drop in maximum forgetting from 42.41 to 7.10 points on the Qwen‑3B MATH stream.

By Zijie Liu, Zhen Tan, Charles Fleming, Tianlong Chen
llmsfine-tuning
More like this →