← Back to all news
arXiv Computation and Language

Automated Detection and Structuring of Social Tipping Point Evidence in Climate related Documents: A Modular AI Framework

The paper introduces an open, modular AI framework that automatically detects and structures evidence of social tipping points in climate literature at the passage level. It integrates a DistilBERT boundary splitter, an iteratively augmented RoBERTa classifier, a Mistral 7B rewrite model, a LLaMA 3.2 3B rating model, and a Milvus vector store, all accessible via a Streamlit interface. Evaluation on a GPT‑4.1‑labelled benchmark and expert‑reviewed set shows the splitter outperforms competitors and the RoBERTa detector achieves high accuracy and agreement.

Related stories

arXiv Machine Learning
Jun 25

Paid Voices vs. Public Feeds: Interpretable Cross-Platform Theme-Based Analysis of Climate Discourse

arXiv:2601. 13317v2 Announce Type: replace-cross Abstract: Climate discourse online shapes public understanding of climate change and informs political and policy debate, yet it unfolds across structurally different environments: paid advertising platforms host targeted, institutionally produced messaging, while public social media reflects largely organic, user-driven discussion.

By Samantha Sudhoff, Pranav Perumal, Zhaoqing Wu, Tunazzina Islam
llms
More like this →
arXiv Computation and Language
Oct 5

Writerslogic at the CLEF 2026 SimpleText Track: Multi-Candidate LLM Simplification and Stacked Complexity Spotting

The Writerslogic team participated in the CLEF 2026 SimpleText shared task, tackling both text simplification (Task 1) and complexity spotting (Task 2). For simplification, they built a multi‑candidate pipeline with GPT‑4o‑mini, selecting the best candidate via a reference‑free heuristic, and their Claude Sonnet 4 submission achieved a SARI of 47.43 and BLEU of 14.21, ranking third overall on the Task 1 leaderboard. For complexity spotting, they fine‑tuned a DeBERTa‑v3‑large NLI model on 350 K labeled pairs, achieving a macro F1 of 0.8081 (0.8085 in an ensemble) on binary over‑generation identification and 0.804 accuracy on multi‑class error classification, placing them second among unique teams.

By David L. Condrey
llmsfine-tuningbenchmarkssafety
More like this →
arXiv AI
6d ago

Can Large Language Models Review the Scientific Literature? A Test Against a 102-Article Benchmark

The study evaluates six large language models (LLMs) on a 102-article benchmark about the Spanish plume weather pattern, testing them across four review stages: article discovery, interpretation, synthesis, and future research proposals. Five tools collectively found 57 relevant articles, with institutional library access boosting discovery for some models. While three tools correctly identified all 18 substantial articles, they misclassified a large portion of passing mentions and produced syntheses that, though largely accurate, contained unsupported claims and quotation errors, and their research proposals were often broader than the benchmark scope.

By Abdullah Al Kalbani, David M. Schultz, Alexandra K. Anderson-Frey, Zhonghua Zheng, Catherine McGuire
llmsbenchmarks
More like this →
arXiv Computation and Language
Aug 28

RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature

RATIO (Retrieval Across Typed Ideation Operations) is a large-scale benchmark designed to evaluate how well retrieval systems can support scientific inspiration. It defines relevance through three ideation moves—Address, Broaden, and Specify—each targeting different levels of abstraction in literature retrieval. The benchmark is built from millions of full-text CS papers using a novel discourse-marker distant supervision method, and includes extensive LLM and human vetting to ensure quality.

By Maayan Sharon, Tom Hope
llmsfine-tuningbenchmarks
More like this →
arXiv AI
Aug 20

Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

The paper investigates how large language models can extract contextualized data from scientific literature. It presents four workflows: expert‑written prompts, self‑generated prompts, autonomous literature discovery, and dataset creation from guidelines. While models perform well with prompts, they struggle with context, hallucinate references, and still need human oversight for final validation.

By Valentin Romanov, Monique Bax, Steven Niederer
llmsagents
More like this →