arXiv Machine Learning
Jun 3
Alignment-Aware Decoding
arXiv:2509. 26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing.
Scaling Unsupervised Word Alignment to Documents via Structural Constraints
Related stories
arXiv Computation and Language
2d ago
STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation
The paper introduces STAR, a metric that measures sentence-level alignment between source and target documents in document-to-document machine translation. Using STAR, the authors develop StarPO, a preference‑optimization framework that ranks translation hypotheses by structural quality and applies a dynamic alignment mask to focus learning on misaligned segments. Experiments on news and literary data show that StarPO improves both translation quality and structural integrity, enabling small models to outperform large proprietary systems such as GPT‑4o while remaining more token‑efficient.
arXiv Computation and Language
2d ago
Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
The study investigates whether monolingual language models, trained without joint multilingual objectives, develop cross-lingual alignment. By evaluating models such as Goldfish and independently built monolingual systems, the authors find that alignable representational geometry emerges across layers, strengthening with larger data, larger models, or closer linguistic proximity. A single Procrustes rotation on parallel sentences can map hidden states between models, and applying this rotation to a German model’s residuals swaps factual predictions to those of the donor English model, demonstrating functional transfer.
arXiv AI
Jul 7
Safe Inference-Time Alignment via Lagrangian Reward Augmentation
arXiv:2607. 02781v1 Announce Type: cross Abstract: Inference-time alignment steers a frozen language model during decoding using auxiliary reward signals, avoiding the cost of repeated weight updates.
arXiv Machine Learning
Jun 5
Multilingual Coreference Resolution via Cycle-Consistent Machine Translation
arXiv:2606. 05444v1 Announce Type: cross Abstract: Coreference resolution is a core NLP task, having a broad range of downstream applications, e.
arXiv Computation and Language
5d ago
Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment -- Is English Enough?
arXiv:2608.03446v2 Announce Type: replace Abstract: Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the giv...
arXiv AI
Jul 7
Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders
arXiv:2607. 04071v1 Announce Type: cross Abstract: Portuguese remains underrepresented in text embedding evaluation, despite being one of the most widely spoken languages in the world.
arXiv Computation and Language
2d ago
Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations
The study compares an English-only and a bilingual decoder-only model, each 310 M parameters, trained on eight diverse languages while controlling for English exposure, compute, and document overlap. After aligning on shared English vocabulary, the authors find that token embeddings appear similar, but the deeper hidden states used for prediction diverge across models. This hidden‑state mismatch grows through middle transformer layers and persists despite controls, indicating that contextual processing differs between the models. "whyItMatters":"The findings show that embedding alignment can conceal significant internal representation differences, which is crucial for any downstream work that assumes aligned multilingual models are interchangeable."
arXiv AI
Jul 9
DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation
arXiv:2607. 07669v1 Announce Type: cross Abstract: Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed.
arXiv AI
Jun 12
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation
arXiv:2606. 13647v1 Announce Type: cross Abstract: We introduce SkMTEB, the first comprehensive MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language, comprising 31 datasets across 7 task types -- nearly 4$\times$ the depth of existing multilingual benchmark coverage for Slovak.
arXiv AI
2d ago
MIMO: Multilingual Information Retrieval via Monolingual Objectives
arXiv:2605.31171v2 Announce Type: replace-cross Abstract: Multilingual Information Retrieval (MLIR) reflects real-world search environments in which queries and relevant documents may appear in diffe...
arXiv AI
Jun 8
Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
arXiv:2602. 07026v3 Announce Type: replace-cross Abstract: Despite the success of multimodal contrastive learning in aligning visual and linguistic representations, a persistent geometric anomaly, the Modality Gap, remains: embeddings of distinct modalities expressing identical semantics occupy systematically offset regions.