This study evaluates whether large language models (LLMs) can reliably replace or complement human research assistants in linguistically complex annotation. In a controlled annotation task, 73 human a...
The paper introduces discourse dependency (DDP) as a continuous measure of translation difficulty based on how far back a segment must look to resolve references. DDP is computed from named entity re‑mentions and pronominal coreference, and is validated against gold coreference with high reliability. Applying DDP to recent WMT benchmarks reveals a bias toward low‑DDP segments, and experiments show that as DDP increases, no current context‑injection strategy matches human post‑editing quality.
By Ahrii Kim, Chanjun Park, Seong-heum Kim
arXiv:2608.28776v1 Announce Type: new
Abstract: Multilingual sentence embeddings are increasingly used to estimate semantic similarity across languages, yet their sensitivity to fine-grained translat...
By Eleftherios Kalogeros, Athanasios Ntalakas, Manolis Gergatsoulis, Paschalis Nikolaou, Sotiria-Lito Alexaki
The paper presents a Japanese discourse parsing system built on Segmented Discourse Representation Theory (SDRT) by translating existing English SDRT corpora with a large language model (LLM). Comparative experiments with prompt-based and fine-tuned LLMs show that Japanese and English parsers achieve similar performance, indicating cross‑lingual applicability of SDRT. Ablation studies reveal that Japanese discourse markers have little influence, while predictions rely mainly on contextual discourse relations rather than clause content, explaining the comparable results across typologically distinct languages.
By Satomu Tomisaki, Yosuke Mikami, Amane Watahiki, Hitomi Yanaka
MetaHOPE is an error‑severity‑aware annotation framework designed to evaluate how well machine translation (MT) and large language models (LLMs) translate metaphors. The authors applied MetaHOPE to three state‑of‑the‑art systems—GoogleMT, GPT5.4, and Hunyuan‑7b—using two human‑annotated metaphor corpora (VUAMC and PSUCMC) for English‑to‑Chinese and Chinese‑to‑English translation. They also produced a bilingual post‑edited gold reference, creating a new resource for metaphor translation research.
By Jiahui Liang, Lifeng Han
The Public Discourse Corpus (PDC) is the first dataset of public‑figure interview speech annotated for affective valence and epistemic modality. It contains 998 videos from 100 speakers across seven professional domains, yielding 186,642 sentences (3.1 million words). A key methodological contribution is Target Speaker Participation (TSP), a five‑category annotation taxonomy with documented inter‑annotator reliability (κ = 0.616), and an audio‑first diarization pipeline that separates target‑speaker turns from interviewer and third‑party speech. The corpus, annotation tools, validation sample, and processing pipeline are released as open source.
By Bo Chen