← Back to all news
arXiv Computation and Language By Jihyung Park, Minchao Huang, Leqi Liu, Elias Stengel-Eskin

PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding

Read the original on arXiv Computation and Language →

PragReST is a self‑supervised framework that enhances large language models’ pragmatic reasoning by generating counterfactual reasoning traces and fine‑tuning models with supervised learning and reinforcement learning, all without human‑labeled data. The method constructs pragmatic QA data, trains models to internalize counterfactual reasoning, and demonstrates significant accuracy gains on four pragmatic benchmarks compared to backbone models and other baselines. Error analysis shows that counterfactual reasoning reduces mistakes from failing to contrast observed utterances with plausible alternatives, and removing it markedly degrades performance while preserving out‑of‑domain general‑knowledge and mathematical reasoning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

Hugging Face Trending Papers
Jun 17

PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding

Natural language understanding often depends on meanings that are implied rather than explicitly stated, requiring pragmatic reasoning. Despite strong performance on math and logical reasoning, large language models (LLMs) still struggle with making pragmatic inferences, often choosing literal interpretations.

llmsreinforcement-learningfine-tuningefficiencybenchmarks
More like this →
arXiv Machine Learning
Aug 28

Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum

The paper investigates whether the high costs of training chain-of-thought reasoning models can be reduced through algorithmic design. It introduces an autocurriculum approach that lets the model select which problems to focus on during training, showing that this method provably improves both supervised fine‑tuning and reinforcement learning. For supervised fine‑tuning, autocurriculum requires exponentially fewer reasoning demonstrations by targeting prompts where the model struggles, while for reinforcement learning it decouples computational cost from the quality of the reference model, making the burn‑in cost nearly independent of target accuracy.

By Nived Rajaraman, Audrey Huang, Miro Dudik, Robert Schapire, Dylan J. Foster, Akshay Krishnamurthy
llmsreinforcement-learningfine-tuning
More like this →