arXiv Computation and Language

CEFR Level Assessment across Languages and Text Granularities

The paper introduces a unified training objective that blends focal loss with supervised contrastive learning to improve automatic CEFR level assessment. This approach addresses class imbalance and ambiguous level boundaries by emphasizing hard examples and learning discriminative representations. Experiments on sentence- and document-level CEFR assessment in both monolingual and multilingual contexts show that the method outperforms strong baselines and sets new state‑of‑the‑art results on multiple benchmarks.

arXiv Computation and Language

Aug 25

LuxIT: A Luxembourgish Instruction Tuning Dataset from Monolingual Seed Data

LuxIT is a monolingual instruction‑tuning dataset for Luxembourgish, created by synthesizing instruction‑answer pairs from native texts using the DeepSeek‑R1‑0528 model and a quality‑assurance LLM‑as‑judge process. The resulting 227,507 high‑quality pairs were used to fine‑tune 14 LLMs (≤15 B parameters), yielding an average accuracy increase of +5.37 percentage points on standardized Luxembourgish proficiency exams and improvements in macro‑averaged F1 on nine of the fourteen downstream NLP tasks. These findings demonstrate that synthetic monolingual data can effectively enhance LLM performance in low‑resource languages and reveal the complex relationship between exam performance and practical NLP gains.

By Julian Valline, Cedric Lothritz, Siwen Guo, Jordi Cabot

llms nlp fine-tuning benchmarks

arXiv Machine Learning

Jun 18

Fair Cognitive Impairment Detection Through Unlearning

arXiv:2606. 18571v1 Announce Type: new Abstract: Mild Cognitive Impairment (MCI) is a medical condition characterized by a noticeable decline in memory, language, or thinking abilities.

By William Nguyen, Jiali Cheng, Hadi Amiri

rag multimodal benchmarks

arXiv AI

Aug 7

Task-Conditional Flow Matching for Balanced Multilingual Text Embedding Adaptation

arXiv:2608. 05785v1 Announce Type: cross Abstract: Multilingual text embedding models are commonly adapted using a single training objective across diverse tasks, despite different tasks requiring fundamentally different optimization strategies.

By Tirth Bhatt, Naren Kumar S, Mayank Singh

rag diffusion benchmarks

arXiv Computation and Language

Oct 8

Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring

arXiv:2610.09660v1 Announce Type: new Abstract: Automatic Short Answer Scoring (ASAS) requires models that can score student responses against question-specific criteria while remaining efficient and...

By Zhifan Sun, Sebastian Gombert, Fabian Zehner, Leon Camus, Longwei Cong, Hendrik Drachsler

llms benchmarks

arXiv Computation and Language

Oct 8

MixedPEFT: Combining Multiple PEFT Methods with Mixed Objectives for Unsupervised Domain Adaptation

arXiv:2606.22272v2 Announce Type: replace Abstract: Applying pre-trained language models to new domains through full fine-tuning is computationally expensive and prone to catastrophic forgetting. To...

By Mohammed Rawhani, Dervi\c{s} Karabo\u{g}a, \"Ozkan Ufuk Nalbanto\u{g}lu, Alper Ba\c{s}t\"urk, Bahriye Akay

llms fine-tuning benchmarks

arXiv AI

Aug 7

Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors

arXiv:2608. 06300v1 Announce Type: new Abstract: Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first language (L1) or age.

By Arya Labroo, Mengjie Qian, Kate Knill

llms rag diffusion multimodal safety

arXiv Machine Learning

Jul 22

Translation as Augmentation: Effect of Translated Data on Assessment of Difficulty

arXiv:2607. 19101v1 Announce Type: cross Abstract: Reliable Text Difficulty Assessment is a prerequisite for valid text simplification workflows and personalized learning applications.

By Yiheng Wu, Jue Hou, Roman Yangarber

nlp

OpenAI Blog

Jun 11, 2018

Improving language understanding with unsupervised learning

We’ve obtained state-of-the-art results on a suite of diverse language tasks with a scalable, task-agnostic system, which we’re also releasing. Our approach is a combination of two existing ideas: transformers and unsupervised pre-training.

llms benchmarks

arXiv Computation and Language

Oct 19

Steering beyond Language: Improving Multilingual Training-Free Text Embeddings via Paired-Residual Subspace Removal

arXiv:2610.23807v1 Announce Type: new Abstract: While deriving text embeddings directly from frozen LLMs has emerged as a promising direction, existing methods focus almost exclusively on English, le...

By Ailiang Lin, Zhuoyun Li, Yusong Wang, Keyu Mao, Kotaro Funakoshi, Manabu Okumura

llms rag

arXiv AI

Jul 7

Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders

arXiv:2607. 04071v1 Announce Type: cross Abstract: Portuguese remains underrepresented in text embedding evaluation, despite being one of the most widely spoken languages in the world.

By Lucas Hideki Takeuchi Okamura, Alexandre Alcoforado, Anna Helena Reali Costa

rag fine-tuning benchmarks

arXiv AI

Oct 7

Cross-Lingual Activation Steering for Multilingual Language Models

arXiv:2601.16390v2 Announce Type: replace-cross Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...

By Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar

llms benchmarks safety

Hugging Face Trending Papers

Sep 2

Unifying Conformal Language Tasks with In-Context Ensembles

The paper introduces the Conformal Relevance framework, which leverages in-context learning example curation and ensembling to generate a score function that preserves coverage while enhancing conciseness for NLP tasks such as summarization and extractive question answering. Unlike traditional methods that rely on labor-intensive, hand-engineered LLM prompts to rate content importance, this approach requires minimal manual input. The authors validate the framework across seven NLP tasks and provide theoretical insights into how diversity in ensembled conformal scores can improve worst-case sentence scores, including a saturation bound on ensemble gains.

llms nlp benchmarks

Related stories