The paper evaluates two large language models, Claude Sonnet 4.5 and Claude Opus 5, on the bidirectional English Resource Grammar (ERG) tasks of generating English from Minimal Recursion Semantics (MRS) and parsing English into MRS. In generation, Opus achieves 76.3 BLEU—surpassing a 72k‑pair trained system and matching a million‑pair system—while Sonnet scores 65.7 BLEU, rising to 69.6 when selecting from ACE’s candidates. In parsing, both models lag behind ACE, attaining only 57.2 and 65.5 F₁ respectively, with exact‑match on about 1 % of sentences, highlighting that high generation scores do not guarantee accurate semantic parsing.
By Soham Dan
arXiv:2609.37883v1 Announce Type: new
Abstract: Pre-trained language models (PLMs) with encoder-based architectures have shown impressive capabilities in zero-shot cross-lingual transfer for various...
By Lalita Lowphansirikul, Attapol Rutherford, Jian Gang Ngui, Sarana Nutanong, Peerat Limkonchotiwat
arXiv:2607. 27421v1 Announce Type: cross Abstract: Intent classification is a core component of task-oriented dialogue systems, yet practitioners have limited systematic guidance for selecting deployable open-weight language models under compute, latency, and robustness constraints.
By Parishruthi Ganesh, Gerry Dozier, Cheryl Seals
arXiv:2609.24650v1 Announce Type: new
Abstract: Readability assessment is essential for tailoring texts to intended audiences across educational, healthcare, and information retrieval domains. Howeve...
By Rapha\"el Thieffry, Matej Martinc
The paper introduces TreeFuse, a training‑free framework that builds a semantic hierarchy from a full document and uses it to select and organize evidence for a query. By combining structural–semantic scoring with lexical attribution, TreeFuse retains supporting facts and relations while focusing the context on the question. Experiments on four large‑context models show a 1.72‑point improvement in QA‑F1 over full‑context prompting and a 58.55% reduction in answer‑context length.
By Zhe Xu, Tianhao Liao, Zhenqing Ling, Ying Shen, Yaliang Li, Daoyuan Chen
FrameBench is a new benchmark that evaluates language models on their ability to distinguish semantic frames evoked by the same verb in different contexts, using multiple-choice questions grounded in FrameNet-style resources for English and Japanese. The dataset is generated and verified through a pipeline that incorporates native-speaker judgments, and the authors provide both the data and the code for construction and evaluation. Experiments show that small models struggle with this task, while several large models outperform human reference scores.
By Chihiro Yano, Ryohei Sasano
GrammarRL introduces a label‑free reinforcement learning approach that adapts language models to grammar constraints without annotated data. It optimizes two self‑supervised rewards—direct and reverse—using a Reinforce Leave‑One‑Out objective over grammar‑constrained rollouts, and regularizes toward a frozen base model. Experiments on sign‑language gloss translation, hierarchical text classification, and named entity recognition with Llama models show consistent gains over constrained greedy decoding and competitive performance to beam search while keeping inference cost low.
By Gabriele Tuccio, Antonino Furnari, Aldo Gangemi, Misael Mongiov\`{\i}
arXiv:2608. 16627v1 Announce Type: cross Abstract: Natural language explanations (NLEs) are increasingly used as inputs, for example, as few-shot rationales that influence model behavior in in-context learning (ICL).
By Mahdi Dhaini, Adam Dejl, Juraj Vladika, Volkan \"Ozer, Barbara Plank, Gjergji Kasneci
arXiv:2606. 02837v1 Announce Type: cross Abstract: Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), making the quality of NL-to-FOL benchmarks essential -- yet these datasets have never been rigorously audited.
By Andrea Brunello, Cristian Curaba, Luca Geatti, Michele Mignani, Angelo Montanari, Nicola Saccomanno
The paper introduces soft‑prompt tuning as a lightweight, automated method for evaluating large language models (LLMs). By optimizing only ten soft‑prompt vectors—about 0.0006% of a 7B model’s parameters—within 150 steps, the approach adapts models to specific benchmark formats, improving format compliance while preserving knowledge accuracy. Experiments across seven models and eleven datasets show that this method matches strong evaluation‑harness baselines, enhances evaluation signals on challenging datasets, and yields cleaner outputs that align better with exact‑match scoring and independent LLM judges.
By Selen Erkan, Bastian Boll, Kristian Kersting, Bj\"orn Deiseroth, Letitia Parcalabescu
arXiv:2605. 31220v2 Announce Type: replace-cross Abstract: Confidence estimation (CE), i.
By Athina Kyriakou, Dennis Ulmer, Ivan Titov
arXiv:2610.16182v1 Announce Type: new
Abstract: Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit t...
By Daoping Wu, Dong Wang, Siyu Liu, Chen Liang, Jian Geng, Ziqi Li, Shuguang Xiong, Ying Zhang