The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The study evaluates self‑supervised pretraining for Prognostics and Health Management (PHM) across 47 public datasets, using a compact Transformer encoder trained with masked patch prediction. Transfer performance is highly task‑dependent: it dramatically improves fault‑classification on CWRU and HSG18 but offers no reliable benefit for remaining‑useful‑life estimation on CMAPSS. The authors also compare centralized versus simulated federated pretraining, finding that centralized training generally outperforms federated training under matched budgets, and that the intrinsic SSL loss does not reliably predict downstream performance.
arXiv:2607. 16681v1 Announce Type: new Abstract: Early sepsis prediction from electronic health records is challenged by irregular sampling, high missingness, and class imbalance.
arXiv:2607. 00958v1 Announce Type: new Abstract: Time series are central to modern data mining applications, from industrial telemetry and server metrics to finance and physiology, yet time-series self-supervised learning often depends on view and augmentation choices that encode domain-specific invariances.
The paper investigates how synthetic tasks influence language‑model pretraining by distinguishing diagnostic, teachable, and transferable signals. In a controlled setting where 70% of the data is general Python and 30% is a mixture of curated synthetic tasks, literature‑derived probes, and OpenCodeInstruct, 14 of 27 tasks show teachability, with curated tasks outperforming literature probes. Curated synthetic data provides conditional value: it boosts downstream performance when mixed with OpenCodeInstruct but is less effective when used alone, and a loss‑based scheduler can misalign short‑term loss reduction with long‑term transfer.
arXiv:2605. 10840v3 Announce Type: replace-cross Abstract: We present Clin-JEPA, a multi-phase co-training framework for joint-embedding predictive (JEPA) pretraining on EHR patient trajectories.
The paper demonstrates that greedy decoding from large language models is not precision‑invariant: the same model, prompt, and decoding algorithm can produce different outputs when run in BF16 versus FP16 on identical hardware. Across six models (1.1B–7B parameters, four families, and 12B) and three benchmarks, 49–100 % of prompts diverge, with a single token flip often cascading into trajectory‑level divergence. The authors develop an empirical error‑propagation analysis that identifies the top‑two logit margin at the LM head as the key factor, and they propose a low‑overhead intervention—selective FP32 LM head recomputation—that improves exact agreement by 22–36 percentage points with less than 4 % latency overhead. "whyItMatters":"The findings reveal that precision choices can fundamentally alter model outputs, challenging the assumption of deterministic greedy decoding and highlighting the need for precision‑aware inference strategies."