As previous research on annotator disagreement in discourse phenomena has shown, understanding text coherence varies considerably from one individual to another. To explore this phenomenon, we created two corpora with multiple annotations of Czech texts, accompanied by annotators' explanations of their choices.
This study evaluates whether large language models (LLMs) can reliably replace or complement human research assistants in linguistically complex annotation. In a controlled annotation task, 73 human a...
The paper introduces discourse dependency (DDP) as a continuous measure of translation difficulty based on how far back a segment must look to resolve references. DDP is computed from named entity re‑mentions and pronominal coreference, and is validated against gold coreference with high reliability. Applying DDP to recent WMT benchmarks reveals a bias toward low‑DDP segments, and experiments show that as DDP increases, no current context‑injection strategy matches human post‑editing quality.
arXiv:2608.28776v1 Announce Type: new Abstract: Multilingual sentence embeddings are increasingly used to estimate semantic similarity across languages, yet their sensitivity to fine-grained translat...
The paper presents a Japanese discourse parsing system built on Segmented Discourse Representation Theory (SDRT) by translating existing English SDRT corpora with a large language model (LLM). Comparative experiments with prompt-based and fine-tuned LLMs show that Japanese and English parsers achieve similar performance, indicating cross‑lingual applicability of SDRT. Ablation studies reveal that Japanese discourse markers have little influence, while predictions rely mainly on contextual discourse relations rather than clause content, explaining the comparable results across typologically distinct languages.
MetaHOPE is an error‑severity‑aware annotation framework designed to evaluate how well machine translation (MT) and large language models (LLMs) translate metaphors. The authors applied MetaHOPE to three state‑of‑the‑art systems—GoogleMT, GPT5.4, and Hunyuan‑7b—using two human‑annotated metaphor corpora (VUAMC and PSUCMC) for English‑to‑Chinese and Chinese‑to‑English translation. They also produced a bilingual post‑edited gold reference, creating a new resource for metaphor translation research.
The Public Discourse Corpus (PDC) is the first dataset of public‑figure interview speech annotated for affective valence and epistemic modality. It contains 998 videos from 100 speakers across seven professional domains, yielding 186,642 sentences (3.1 million words). A key methodological contribution is Target Speaker Participation (TSP), a five‑category annotation taxonomy with documented inter‑annotator reliability (κ = 0.616), and an audio‑first diarization pipeline that separates target‑speaker turns from interviewer and third‑party speech. The corpus, annotation tools, validation sample, and processing pipeline are released as open source.
The paper investigates the patterns of disagreement in emotion annotation by analyzing multilabel annotations from 1,677 sentence-level units of Hemingway’s *The Old Man and the Sea* and comparing them with individual-level annotations from BRIGHTER and GoEmotions. It identifies three recurring disagreement structures: systematic overlap of emotion categories, differences in label breadth and boundary treatment, and sparse participation or abstention that mimic disagreement. The study shows that hard aggregation obscures these patterns, while annotator vote-share representations preserve only marginal frequencies, and it proposes a vocabulary for describing disagreement structure.
arXiv:2510.12306v4 Announce Type: replace Abstract: As natural language corpora expand at an unprecedented rate, manual annotation remains a significant methodological bottleneck in corpus linguistic...
DiscoSign is a new computational framework that translates text into sign language glosses while explicitly handling discourse-level phenomena. It focuses on three key aspects: spatial coreference resolution to keep entities in consistent spatial positions, Question-Answer Clauses (QACs) that serve specific discourse functions, and concept-gloss consistency to maintain stable mappings between English concepts and ASL signs. The authors introduce novel evaluation metrics tailored to these discourse dimensions and demonstrate that DiscoSign improves spatial consistency and entity tracking compared to sentence-only translation models, without sacrificing single-sentence gloss quality.
The study examines how varying the context window size of GPT‑2 affects its alignment with human reading-time data. A U‑shaped relationship is found: very short windows (<20 tokens) capture local memory limits, while larger windows (500–1,000 tokens) provide the best overall fit. Counterfactual experiments that disrupt cross‑sentential coreference chains show that these long‑range coreference relations are crucial for the improved fit of larger context windows.
arXiv:2606.12186v2 Announce Type: replace Abstract: Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjec...
Enthymemes, arguments with unstated premises or conclusions, are pervasive in persuasive discourse, yet their annotation remains notoriously subjective. We present a resource of 1,482 tweets from politically controversial discourse, annotated by five annotators for the presence of enthymemes and their argument structure, designed to study label variation.