arXiv Computation and Language

Inferring L1 Backgrounds from L2 English with LLM Token Probabilities

The paper presents a method for inferring a speaker’s first language (L1) from their second‑language English (L2) by using token probability scores from large language models (LLMs). By treating each text as a continuation of a prompt that specifies a candidate L1, the authors identify the L1 with the highest probability score. Experiments with Llama 3.1 8B and 70B on TOEFL9, Write and Improve 2024, and BULATS‑L1 datasets show that token‑probability scoring outperforms direct classification, with the 70B model achieving state‑of‑the‑art results across all datasets.

arXiv AI
Oct 27

Robust and Fine-Grained Detection of AI Generated Texts

arXiv:2504.11952v4 Announce Type: replace-cross Abstract: An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existe...

By Ram Mohan Rao Kadiyala, Siddartha Pullakhandam, Kanwal Mehreen, Drishti Sharma, Siddhant Gupta, Jebish Purbey, Ashay Srivastava, Subhasya TippaReddy, Arvind Reddy Bobbili, Suraj Telugara Chandrashekhar, Ankan Ghosh, Modabbir Adeeb, Srinadh Vura, Suman Debnath, Hamza Farooq
arXiv AI
Jul 24

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text

arXiv:2607. 21458v1 Announce Type: new Abstract: The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents.

By Yangjun Lu, Hongyi Zhou, Fabian Spill, Kai Ye, Chengchun Shi, Jin Zhu
arXiv Computation and Language
Oct 16

No Single Tokenizer Feature Reliably Predicts Downstream Language Model Performance

The paper investigates whether intrinsic tokenizer features can predict the performance of downstream language models. By training about 200 models across nine languages and evaluating numerous tokenizer metrics, the authors find that none of the existing metrics consistently forecasts model performance across different families, tasks, and languages. They propose a new linguistically grounded metric that captures aspects of tokenizer quality missed by prior measures, though its predictive power may still vary.

By Antonio Lopardo, Luis Frentzen Salim, Shikhar Srivastava, Kevin Jiang, Mridul Sharma, Catherine Arnett
arXiv Computation and Language
Sep 25

A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition

The paper introduces a token‑level extension of Omni‑Temporal Classification (OTC) for automatic speech recognition, allowing unsupported tokens to be bypassed while preserving supervision for the rest of the word. Across 19 languages and three corpora, this token‑level OTC consistently outperforms standard CTC, achieving the lowest mean word error rate on every dataset and a 9.45% average relative WER reduction. A predictive‑entropy‑indexed schedule replaces epoch‑based relaxation, reducing training‑length dependence while maintaining performance.

By Saurabh Kumar, Diptiman Mohanta, Prasanta Kumar Ghosh