arXiv Computer Vision
6d ago

OlmoEarth v1.3: Embeddings

arXiv:2610.32294v1 Announce Type: new Abstract: We make some OlmoEarth embeddings. To do this, we train a new model: OlmoEarth v1.3. OlmoEarth v1.3 is designed to condense multimodal, multitemporal r...

By Gabriel Tseng, Favyen Bastani, Patrick Alan Johnson, Yawen Zhang, Piper Wolters, Ando Shah, Hadrien Sablon, Joseph Redmon, Heather DeHaven, Rachel Ratner, Sasha Levy, Jordan Steward, Katrina Midgley, Andrew Howe, Hunter Pitelka, Ted Schmitt
arXiv AI
Jun 9

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

arXiv:2606. 07524v1 Announce Type: cross Abstract: The explosive growth of large language models (LLMs) has created a heterogeneous and poorly documented ecosystem, making systematic model comparison increasingly important for provenance auditing, security analysis, and model selection.

By Zirui Wang, Yusen Hou, Shaofeng Liang, Bowen Tian, Yanlin Zhang, Wenshuo Chen, Yutao Yue
arXiv AI
Oct 15

Scaling Point-in-Time Language Models: Enabling Economic Evaluation of Embeddings

The paper introduces point‑in‑time language models that are trained only on text available up to each calendar date, thereby eliminating lookahead bias that plagues standard large language models. By scaling decoder‑only transformers to 4 billion parameters and training on 1 trillion chronologically filtered tokens from FineWeb, the authors produce monthly checkpoints from 2013 to 2024 that achieve performance close to unconstrained models of similar size on common‑sense reasoning and language understanding tasks. Instruction fine‑tuning with LoRA further enhances downstream usability, and the authors release the full pipeline—including dataset construction, training infrastructure, and evaluation code—for reproducible research.

By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu