arXiv Machine Learning

Fitting Large Nonlinear Mixed Effects Models Using Variational Expectation Maximization

The paper introduces a scalable Variational Expectation Maximization (VEM) algorithm for fitting large Nonlinear Mixed Effects (NLME) models, addressing computational challenges that arise as parameter and random‑effect counts grow. VEM leverages flexible variational families and reverse‑mode automatic differentiation to efficiently maximize the marginal likelihood, enabling fitting of models with over 15,000 population parameters. Experiments using the Pumas software demonstrate VEM’s ability to improve log‑likelihood over many iterations while remaining computationally feasible, whereas traditional FOCE methods fail to complete even a single iteration for similarly sized models.

arXiv Computation and Language
1d ago

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters...

By Hao Liang, Zhengyang Zhao, Mingrui Chen, Meiyi Qiang, Lu Ma, Rongyi Yu, Hengyi Feng, Shixuan Sun, Zimo Meng, Xiaochen Ma, Xuanlin Yang, Qifeng Cai, Ruichuan An, Bohan Zeng, Zhen Hao Wong, Chengyu Shen, Runming He, Zhaoyang Han, Yaowei Zheng, Fangcheng Fu, Conghui He, Bin Cui, Zhiyu Li, Weinan E, Wentao Zhang
arXiv Machine Learning
Aug 20

Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference

Monroe is a new molecular foundation model that improves upon existing models by pre‑training on over 81 million molecules from the PM6 quantum chemistry dataset, enhancing stereochemistry representation, and introducing novel training losses such as conformer denoising and embedding decorrelation. It also incorporates a prior‑data‑fitted model (TabPFN) for downstream in‑context prediction and demonstrates superior performance on Polaris benchmarks and activity cliff tests. Ablation studies show that the PFN‑based downstream approach can upgrade other models, producing state‑of‑the‑art variants MiniMol_PFN and CheMeleon_PFN.

By Blazej Banaszewski, Andrew W. Fitzgibbon