arXiv Computation and Language
Sep 25

How To Do Things With Prompts

The paper examines how users’ prompts to large language models evolve over time, applying speech act and politeness theory to a corpus of 2,000 English prompts from 2023 and 2025. It finds a shift toward more indirect, implicit, and fragmentary directive speech acts, with a notable 14.9‑percentage‑point drop in explicit propositional content and a decline in politeness markers. This suggests users increasingly rely on the model’s inferential abilities, treating it as a competent implicature resolver.

By Kristina \v{S}ekrst, Virna Karli\'c
arXiv AI
Sep 3

VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages

VakyArth is the first pragmatic benchmark for Indic languages, covering Hindi, Punjabi, Tamil, and Malayalam. It tests models on five pragmatic phenomena—deixis, speech acts, implicature, social pragmatics, and coherence—using multiple-choice questions, natural language inference, and translation tasks authored by native speakers. Evaluation of multilingual LLMs shows consistent failures on pragmatic meanings rooted in Indic linguistic and cultural conventions, with systematic differences across languages and tasks.

By Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda, Anand Madhusoodanan, Geyang Guo, Wei Xu, Junyi Jessy L
arXiv Computation and Language
Oct 28

Japanese Discourse Structure Analysis Based on Segmented Discourse Representation Theory Using LLMs

The paper presents a Japanese discourse parsing system built on Segmented Discourse Representation Theory (SDRT) by translating existing English SDRT corpora with a large language model (LLM). Comparative experiments with prompt-based and fine-tuned LLMs show that Japanese and English parsers achieve similar performance, indicating cross‑lingual applicability of SDRT. Ablation studies reveal that Japanese discourse markers have little influence, while predictions rely mainly on contextual discourse relations rather than clause content, explaining the comparable results across typologically distinct languages.

By Satomu Tomisaki, Yosuke Mikami, Amane Watahiki, Hitomi Yanaka
arXiv Computation and Language
Oct 12

Not Just an Evaluation: A Case Study of Discourse Particle Sensitivity in Language Models

The paper studies how large language models (LLMs) handle the discourse particles "just" and "only" in natural conversation. Behavioral experiments show that both humans and LLMs prefer these particles and are more sensitive to "just" than "only", while instruction tuning enhances this sensitivity. Mechanistic analysis identifies specific units in the models that respond to each particle, revealing an asymmetric overlap in their internal representations.

By William Sheffield, Jiahui Chen, Amy Zhang, Alex Warstadt, Junyi Jessy Li, Kanishka Misra
arXiv AI
Oct 16

RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing

RAG‑RST is a retrieval‑augmented framework that supplies a formal discourse schema as in‑context demonstrations for Rhetorical Structure Theory parsing. By retrieving exemplar EDU pairs, it improves nuclearity F1 from 0.794 to 0.846 and relation F1 from 0.738 to 0.788 without any training. Two configurations—precomputed and dynamic—both outperform existing RST parsers, with the dynamic variant consistently strongest, and the approach shows that a formal linguistic schema can be provided to a foundation model as a demonstration rather than encoded in its parameters.

By Mohammadreza Sediqin, Shlomo Engelson Argamon