arXiv:2608. 05980v1 Announce Type: new Abstract: We investigate whether simple transformations can translate representations across heterogeneous text embedding models.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.
arXiv:2608. 05785v1 Announce Type: cross Abstract: Multilingual text embedding models are commonly adapted using a single training objective across diverse tasks, despite different tasks requiring fundamentally different optimization strategies.
arXiv:2607.00171v2 Announce Type: replace Abstract: Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover on...
arXiv:2606. 13647v1 Announce Type: cross Abstract: We introduce SkMTEB, the first comprehensive MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language, comprising 31 datasets across 7 task types -- nearly 4$\times$ the depth of existing multilingual benchmark coverage for Slovak.
The paper investigates how to effectively pre‑train language models when the data budget is limited but compute is plentiful. It shows that increasing model size only improves performance up to an optimal point, after which overfitting degrades generalization, and that this optimal size varies with both the data budget and downstream tasks. To overcome the inefficiencies of standard Transformers in this regime, the authors propose recursive Transformers that reuse a shared block across depth and employ factorized embeddings, achieving better results than standard models on 10M–100M word pre‑training budgets and competitive performance with BabyLM Challenge 2025 winners.
arXiv:2603.18908v5 Announce Type: replace Abstract: Independently trained language models often learn compatible late-stage representations, despite differences in training objectives, architectures,...
The paper challenges the assumption that improving cross‑lingual alignment automatically enhances cross‑lingual transfer. Using XLM‑R models aligned on token, sentence, and masked‑language‑modeling objectives across four language pairs, the authors evaluate zero‑shot transfer on part‑of‑speech tagging and sentence classification. They find that embedding‑based alignment metrics poorly predict downstream performance and that alignment and task gradients are often nearly orthogonal, especially when operating at different representational levels.