SciTBERT is a family of BERT-derived language models trained on chronologically consistent corpora of scientific papers, patents, and educational web text, with training cutoffs spanning 2013 to 2025. The models are further post‑trained using paper and patent citations to create the SciTBERT‑CI family, which better captures temporal dynamics. Experiments show that these chronologically consistent encoders outperform earlier domain‑specific models, especially when training data is limited by early year restrictions, and they perform well on the new PatRepEval benchmark of patent‑related tasks at the science‑technology interface.
By Thomas Gebhart, Russell J. Funk
arXiv:2606. 18801v1 Announce Type: cross Abstract: With the rapid expansion of massive multilingual corpora, Multilingual Information Retrieval (MLIR) has emerged as a critical technology for global information access.
By Youngjoon Jang, Seongtae Hong, Hyeonseok Moon, Heuiseok Lim
BioELX is a retrieve‑rerank framework for cross‑lingual biomedical entity linking that tackles two key problems: the English‑biased UMLS alias training data and the degradation caused by naïvely adding context. It fine‑tunes SapBERT_multi with Wikidata‑derived cross‑lingual alias supervision to create shared concept neighborhoods, and then reranks candidates using pretrained LLMs with mention‑anchored prompting to focus on the target mention. Experiments demonstrate state‑of‑the‑art performance on four benchmarks, improving Recall@1 by 4.8–18.2 percentage points without task‑specific annotations.
By Yi Wang, Corina Dima, Liangyu Zhong, Steffen Staab
arXiv:2604. 19047v2 Announce Type: replace-cross Abstract: Existing QA benchmarks typically assume distinct documents with minimal overlap, yet real-world retrieval-augmented generation (RAG) systems operate on corpora such as financial reports, legal codes, and patents, where information is highly redundant and documents exhibit strong inter-document similarity.
By Hanjun Cho, Jay-Yoon Lee
The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.
By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
The paper presents a method for improving cross‑lingual retrieval in multilingual encoders by adding a query‑only low‑rank adapter and deterministic routing based on language. The adapter, trained against frozen document embeddings, is used for cross‑lingual queries while the original encoder handles same‑language queries, preserving existing performance. Using the SampoTron LoRA adapter with the Nemotron‑3‑Embed‑1B model, the authors report a 20.9% relative gain in nDCG@10 across six English, Finnish, and Swedish directions on a financial benchmark, with all cross‑lingual directions improving and same‑language performance maintained.
By Akshay Jain, Edward Kim
The paper investigates cross‑lingual transfer for sequential sentence classification (SSC) in research papers, focusing on 13 non‑English languages. Experiments show that linguistic proximity does not reliably predict transfer success, whereas structural similarity in rhetorical organization—particularly label distribution similarity—correlates positively with performance. The authors introduce three generative‑model methods that exploit structural cues, achieving parity with strong encoder baselines on‑domain and outperforming them when transferring to unseen languages.
By Kazuhiro Yamauchi, Marie Katsurai
arXiv:2606. 24200v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) in clinical settings increasingly requires multilingual retrieval against predominantly English evidence corpora.
By Junhyeok Lee, Han Jang, Hyeonjin Goh, Kyu Sung Choi
arXiv:2606. 04382v1 Announce Type: cross Abstract: Automated subject cataloging assigns controlledvocabulary headings to bibliographic records, but LCSH has no standard public benchmark.
By Kwok Leong Tang
The paper introduces a three‑stage training pipeline that builds compact, efficient dense retrievers without requiring ground‑truth relevance labels. Using cross‑lingual alignment, relational knowledge distillation, and contrastive fine‑tuning, the authors develop PolDense (six Polish models ranging from 17 M to 1 B parameters) and EuroDense (a 435 M‑parameter model covering nine European languages). Extensive evaluation on 41 Polish and 150 multilingual tasks shows that PolDense‑1B outperforms larger retrievers up to 9 B parameters, while EuroDense leads in task‑averaged and language‑averaged performance among models below 1 B parameters.
By S{\l}awomir Dadas, Rafa{\l} Po\'swiata, Ma{\l}gorzata Gr\k{e}bowiec, Micha{\l} Pere{\l}kiewicz
The paper introduces a multilingual, multi-functional framework for disambiguating funder names in scientific publications, using a training dataset that merges the Research Organization Registry with Web of Science and Crossref Open Funder Registry data. By applying multi-task learning with contrastive and multiple negatives ranking losses, the authors fine‑tune open‑weight embedding models from the Sentence Transformer, Gemma, and Qwen3 families, achieving over 90% accuracy in matching Web of Science funder names to ROR identifiers and surpassing general‑purpose LLMs by more than 0.1. For funders not present in ROR, a similarity network is constructed to identify clusters, and the study discusses challenges related to smaller and non‑English‑speaking funders.
By Kanyao Han, Zhiwen You, Jinseok Kim, Jana Diesner
arXiv:2608. 02112v1 Announce Type: new Abstract: Embedding benchmarks measure standalone model quality, but they do not establish whether a low-cost retriever contributes complementary ranking information once lexical and transformer-based retrieval are already combined.
By Ant\'onio Pereira Barata