arXiv AI August 12, 2026 By Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman, Guy Leib, Omri Nayshool, Dan Ofer, Tal Zinger, Dan Dominissini, Gideon Rechavi, Nicole Bussola, Simon Lee, Shane O'Connell, Dung Hoang, Marissa Wirth, Alexander W. Charney, Nati Daniel, Yoli Shavit

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

Read the original on arXiv AI →

arXiv:2602. 17162v3 Announce Type: replace Abstract: Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature".

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 10

Entropy, Disagreement, and the Limits of Foundation Models in Genomics

arXiv:2604. 04287v2 Announce Type: replace Abstract: Foundation models in genomics have shown mixed success compared to their counterparts in natural language processing.

By Maxime Rochkoulets, Lovro Vr\v{c}ek, Mile \v{S}iki\'c

rag nlp

arXiv AI

Jun 6

Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning

arXiv:2606. 05173v1 Announce Type: cross Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper semantic structure.

By Aimen Boukhari

rag diffusion benchmarks

arXiv Machine Learning

Aug 3

PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction

arXiv:2607. 29378v1 Announce Type: cross Abstract: Large language models (LLMs) generate text by auto-regressively sampling the next token.

By Pirzada Suhail, Nagasai Saketh Naidu, Atanu R Sinha, Amit Sethi

llms fine-tuning

arXiv Machine Learning

Jun 16

Emergent retokenization symmetry in large language models: phenomenology and applications

arXiv:2606. 15521v1 Announce Type: cross Abstract: Tokenization introduces representational redundancy: under a fixed token vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string.

By Kanishk Jain, Matthew Day, Tankut Can

llms computer-vision nlp benchmarks

arXiv AI

Jul 1

Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models

arXiv:2410. 12341v4 Announce Type: replace-cross Abstract: As AI-generated content increasingly populates the web, generative AI models are at growing risk of being trained on their own outputs, a process known as AI autophagy.

By Daniele Gambetta, Gizem Gezici, Fosca Giannotti, Dino Pedreschi, Alistair Knott, Luca Pappalardo

llms fine-tuning

arXiv Machine Learning

Jun 16

Next-Latent Prediction Transformers Learn Compact World Models

arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.

By Jayden Teoh, Manan Tomar, Kwangjun Ahn, Edward S. Hu, Tim Pearce, Pratyusha Sharma, Akshay Krishnamurthy, Riashat Islam, Alex Lamb, John Langford

llms diffusion benchmarks safety

Related stories

Entropy, Disagreement, and the Limits of Foundation Models in Genomics

Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning

PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction

Emergent retokenization symmetry in large language models: phenomenology and applications

Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models

Next-Latent Prediction Transformers Learn Compact World Models