arXiv Machine Learning

IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems

arXiv:2607. 23322v1 Announce Type: cross Abstract: Instruction tuning has become the standard method for adapting large language models to follow human intent, yet existing instruction datasets are dominated by English-language general-knowledge tasks and lack coverage of specialized pedagogical domains.

arXiv Machine Learning
Aug 18

L3Cube-IndicQuest v2: A Large-Scale Multilingual Benchmark for Evaluating Factual Knowledge of Large Language Models Across Indic Languages

arXiv:2608. 15535v1 Announce Type: cross Abstract: We present L3Cube-IndicQuest v2, a large-scale gold-standard multilingual question-answering benchmark for evaluating the India-specific factual knowledge of Large Language Models (LLMs).

By Rinit Jain, Tirthraj Mahajan, Advait Joshi, Raviraj Joshi
arXiv Computation and Language
Sep 1

Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation

The paper investigates how to choose language models (teachers) for generating multilingual synthetic data used to fine‑tune smaller student models. By evaluating 10 teacher models across six diverse languages and training 240 students, the authors find that teacher effectiveness is not driven by model size but by data qualities such as prompt diversity, length, and fluency, which explain most of the variance in student performance. Practical guidelines are offered, including matching teacher and student families and using translated prompts to improve outcomes for low‑resource languages.

By Lester James V. Miranda, Ivan Vuli\'c, Anna Korhonen
arXiv Computation and Language
Sep 18

Viveka-Insight: a cross-lingual concept graph and citation-grounded retrieval resource over the complete works of Swami Vivekananda in English and Bengali

Viveka-Insight is a bilingual resource and open‑source pipeline for Swami Vivekananda’s complete works, providing a structure‑preserving parse of 32,694 paragraphs and 168,842 sentences, a cross‑lingual concept graph with 8,362 language‑agnostic concepts, a bilingual alias inventory of 60,850 surface forms, and a human‑annotated set of 200 paragraph‑concept edges. The resource enables citation‑grounded retrieval across the English and Bengali corpora, achieving Recall@10 of 0.86 for known‑item cross‑lingual queries and demonstrating concept‑extraction precision of 0.60 (up to 0.71 with confidence filtering). The design is intended to be transferable to other multilingual classical corpora.

By Tamal Maharaj
arXiv AI
Sep 3

VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages

VakyArth is the first pragmatic benchmark for Indic languages, covering Hindi, Punjabi, Tamil, and Malayalam. It tests models on five pragmatic phenomena—deixis, speech acts, implicature, social pragmatics, and coherence—using multiple-choice questions, natural language inference, and translation tasks authored by native speakers. Evaluation of multilingual LLMs shows consistent failures on pragmatic meanings rooted in Indic linguistic and cultural conventions, with systematic differences across languages and tasks.

By Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda, Anand Madhusoodanan, Geyang Guo, Wei Xu, Junyi Jessy L
arXiv AI
Sep 3

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

The paper introduces DKL, a method for adding new knowledge to instruction‑tuned language models without compromising their instruction‑following abilities. DKL performs extended pre‑training on a base LLM to embed knowledge, then merges these weights into the instruction‑tuned model, avoiding costly instruction fine‑tuning. Experiments show DKL raises RAG accuracy from 54.17% to 79.26% on retrieval failure cases while using far less training data than previous approaches.

By Kushagra Bhushan, Meghanadh Pulivarthi, Sai Krishna Reddy Sathi, Gaurav Pandey, Sonam Gupta, Vineet Kumar, Jaydeep Sen, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
Hugging Face Trending Papers
Sep 2

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

The paper introduces DKL, a method that decouples knowledge learning from instruction tuning in language models. Instead of fine‑tuning the instruction‑tuned model directly, DKL first extends pre‑training on a base model to embed new knowledge, then merges these weights into the instruction‑tuned model, preserving its instruction‑following abilities. Experiments show DKL raises RAG accuracy from 54.17 % to 79.26 % on retrieval failures, outperforming prior methods while using far less training data.