arXiv AI By Hugo Math, Maximilian Forstenh\"ausler, Christos Anagnostopoulos

DES-JEPA: Learning a Shared Vocabulary for High-Dimensional Discrete Event Sequences

Read the original on arXiv AI →

DES-JEPA introduces a joint-embedding predictive architecture that learns a shared discrete vocabulary from synthetic stochastic processes with unaligned event vocabularies. Coupled with DES-PFN, a prior‑fitted network that reuses this vocabulary for in‑context next‑event prediction, the framework can handle up to 700 distinct event types without task‑specific fine‑tuning. It achieves competitive zero‑shot performance on eight unseen real‑world datasets, addressing the limitations of current foundation models for high‑dimensional discrete event sequences.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers
Oct 13

Foundation Models for Event Prediction Across Heterogeneous Event Logs

The paper introduces the Foundation Event Predictor (FEP), a scalable in‑context architecture that performs next‑event prediction across diverse event logs using learned, event‑level nonparametric inference. FEP employs causal self‑attention to encode context and target histories, cross‑attention for relevance‑weighted aggregation, and permutation‑robust embedding learning to decouple event IDs from semantics. Trained on 696 real‑world datasets and evaluated on 50 unseen ones, FEP generalizes effectively to new systems without parameter adaptation, often matching or surpassing models trained separately on each target dataset.

arXiv Machine Learning
Sep 3

A Unified Particle Filter LSTM for Data-Driven Process Simulation

The paper introduces a Unified Particle Filter LSTM (Unified PF‑LSTM) for data‑driven process simulation, which maintains a weighted set of recurrent‑state hypotheses to better capture latent process conditions from incomplete event logs. By summarizing this particle belief with a weighted mean and moment‑generating‑function features, the model predicts next‑activity probabilities and conditional sojourn‑time quantiles. Experiments on three real‑world emergency department datasets show that the framework consistently outperforms existing data‑driven baselines in reproducing routing, duration, and system‑level behavior, especially when process dynamics are only partially reflected in the logs.

By Parvin Malekzadeh, Opher Baron, Dmitry Krass
arXiv Machine Learning
Jul 20

LLM4EHR: Aligning Clinical Time Series with Medical Event Sequences via Large Language Models

arXiv:2607. 15447v1 Announce Type: new Abstract: Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods.

By Jingteng Li, Alexander Capstick, Louise Rigny, Iona Biggart, Neil J Sebire, Payam Barnaghi
arXiv AI
Sep 10

NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

NOAH is a generative transformer that learns the full multimodal patient journey by integrating bidirectional time and a variational latent space. Trained on over 559 million clinical events from 431,000 hospital visits, it processes medical images, time‑series, numeric signals, categorical events, and both structured and unstructured records. The model supports autoregressive forecasting, zero‑shot classification, and counterfactual intervention simulation, yielding strong performance on clinical outcomes, ICD chapters, comorbidities, and time‑to‑event prediction.

By Tobias Susetzky, Raphael Rehms, Dmitrii Seletkov, \"Ozg\"un Turgut, Michelle Espranita Liman, Lisa Steinhelfer, Rickmer Braren, Daniel Rueckert