arXiv Computation and Language

Assessing Readability with LLMs: The Role of Reasoning and Few-Shot Prompting

Hugging Face Trending Papers
Jun 2

From Script to Semantics: Prompting Strategies for African NLI

Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI benchmark.

arXiv AI
Jul 8

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

arXiv:2607. 05992v1 Announce Type: cross Abstract: Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites.

By Daryna Dementieva, Nikolay Babakov, Kathy H\"ammerl, Ilseyar Alimova, Jind\v{r}ich Libovick\'y, Shu Okabe, Miras Baisbay, Lukas Edman, Abrorkhon Inomkhujaev, Antonia Karamolegkou, Mateusz Lango, Volkan \"Ozer, Nikola Selic, Subhankar Swain, Tsedeniya Kinfe Temesgen, Galit Bary Weisberg, Alexander Fraser
arXiv Computation and Language
Sep 10

Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning

arXiv:2609.10445v1 Announce Type: new Abstract: Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: m...

By Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca, Daniel D'souza, Alexandre Berard, Thomas Euyang, Marzieh Fadaee, Julia Kreutzer
arXiv Computation and Language
Sep 2

PromptNCE: Conditional Probabilities and PMI Using Only LLMs and Contrastive Estimation Prompts

The paper introduces PromptNCE, a zero‑shot method that uses large language models to estimate pointwise mutual information (PMI) by framing conditional probability estimation as a contrastive task with an explicit OTHER category. The authors benchmark PromptNCE against four other prompting‑based estimators on three human‑annotated datasets, finding that PromptNCE achieves the best conditional probability estimates and Spearman correlations up to 0.78 for full PMI. A case study demonstrates the method’s utility for scoring student knowledge summaries in low‑data settings, and the authors release code and prompts for reproducibility.

By Juliette Woodrow, Chris Piech