← Back to all news
arXiv Computation and Language By Guillem Ram\'irez

Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

Read the original on arXiv Computation and Language →

The paper introduces SALT, a lightweight post‑training technique that injects span‑level supervision into existing cross‑lingual sentence encoders to enhance token representations. Evaluated on five multilingual token‑level benchmarks, SALT achieves the best overall results on four tasks, surpassing alternative fine‑tuning methods and competitive encoders. Additionally, SALT improves sentence‑level performance on cross‑lingual retrieval and classification tasks, demonstrating the effectiveness of span‑level supervision for both token and sentence representations.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
Aug 27

Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

The paper investigates how to fairly compare language models across languages, noting that current evaluation methods vary widely and lack empirical validation. By training controlled monolingual models on parallel data and testing multilingual LLMs, the authors find that many normalized metrics suffer from biases due to tokenization, encoding, and orthographic differences. Instead, they recommend using sentence‑level negative log‑likelihood over semantically equivalent sequences for more reliable cross‑lingual comparisons.

By Xiulin Yang, Ethan Gotlieb Wilcox, Catherine Arnett
llmsnlp
More like this →
Hugging Face Trending Papers
Jul 20

Token-Level Off-Policy Learning for Faithful Generation Under Distribution Shift

We propose Token-Level Off-Policy Labeling (TOPL), an off-policy training paradigm that reframes post-training as a token-level correctness prediction task. Our key intuition is that by training the model to distinguish good and bad tokens in a response, we naturally guide the model towards generating good tokens, while avoiding the pitfalls that come with directly training the model to generate off-policy tokens.

nlpfine-tuningbenchmarks
More like this →