Hugging Face Trending Papers
SHELF is a Python system that creates controlled benchmark data and evaluation tasks for libraries and archives, using labelled taxonomies, writing specifications, and a generation budget. It generates 62,899 model-written documents based on Library of Congress vocabularies and supports tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. The release compares various methods—including TF, TF-IDF, BM25, popular encoders, and zero-shot decoders—showing that sparse methods remain competitive on classification and that SHELF can vary bibliographic facets independently while generating new, verifiably unseen documents.
SHELF is a Python system that creates synthetic, controlled benchmark data for evaluating large language models on bibliographic tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. It generates 62,899 model-written documents based on Library of Congress vocabularies and compares methods like TF, TF‑IDF, BM25, popular encoders, and zero‑shot decoders, reporting performance metrics such as 0.8887 for subject classification and 0.2605 for genre‑form classification. The tool also allows independent variation of bibliographic facets and can produce unseen documents beyond a model’s training cutoff, with results indicating that model rankings transfer more reliably than absolute scores when compared to other benchmarks.
By Michael J. Bommarito II
arXiv:2606. 04382v1 Announce Type: cross Abstract: Automated subject cataloging assigns controlledvocabulary headings to bibliographic records, but LCSH has no standard public benchmark.
By Kwok Leong Tang
arXiv:2506.17585v4 Announce Type: replace
Abstract: Large language models can answer questions correctly and still hallucinate their citations. Most citation systems address this by retrieving docume...
By Yukun Huang, Sanxing Chen, Jian Pei, Manzil Zaheer, Bhuwan Dhingra
The Writerslogic team participated in the CLEF 2026 SimpleText shared task, tackling both text simplification (Task 1) and complexity spotting (Task 2). For simplification, they built a multi‑candidate pipeline with GPT‑4o‑mini, selecting the best candidate via a reference‑free heuristic, and their Claude Sonnet 4 submission achieved a SARI of 47.43 and BLEU of 14.21, ranking third overall on the Task 1 leaderboard. For complexity spotting, they fine‑tuned a DeBERTa‑v3‑large NLI model on 350 K labeled pairs, achieving a macro F1 of 0.8081 (0.8085 in an ensemble) on binary over‑generation identification and 0.804 accuracy on multi‑class error classification, placing them second among unique teams.
By David L. Condrey
arXiv:2610.02516v1 Announce Type: cross
Abstract: Zero-shot classifiers are useful for routing user requests to specialized LLM tasks, but scoring every request against a large candidate set is expen...
By Haifeng Wu, Srinivasan Manoharan, Jian Wan, Fangbo Tu, Junhua Zhao, Xin Chen
arXiv:2610.16182v1 Announce Type: new
Abstract: Flagship large language models (LLMs) offer rich general knowledge and strong reasoning capabilities, but their high inference cost and latency limit t...
By Daoping Wu, Dong Wang, Siyu Liu, Chen Liang, Jian Geng, Ziqi Li, Shuguang Xiong, Ying Zhang
arXiv:2607. 16848v1 Announce Type: cross Abstract: Long-term memory is becoming a core component of LLM agents, but most memory benchmarks evaluate conversations or compact summaries, while research agents need to restore evidence from full scientific papers.
By Maksim Sheverev, David Finkelstein, Sergey Nikolenko
arXiv:2608.18097v2 Announce Type: replace
Abstract: We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet...
By Amr Sobhy
arXiv:2605.05726v2 Announce Type: replace
Abstract: As LLM agents are increasingly deployed with large libraries of reusable skills, selecting the right skill for a user request has become a critical...
By Hongcheol Cho, Ryangkyung Kang, Youngeun Kim
The paper compares Complement Naive Bayes (NB) with zero‑shot and few‑shot large language models (LLMs) across a wide range of model sizes and text classification tasks. NB outperforms LLMs when labeled data is available, achieving comparable accuracy to large LLMs while running thousands of samples per second on a CPU. In zero‑data sentiment settings, LLMs still dominate, but NB remains the best choice for resource‑constrained HPC practitioners, and the authors provide a Kubernetes Helm operator to automate model selection.
By Mohammad Firas Sada, Dmitry Mishin, John Graham, Seungmin Kim, Mahidhar Tatineni, Frank W\"urthwein
The paper introduces STAIR, a retrieval system that uses a document’s Table of Contents to guide large language models in accessing global structure, thereby reducing hallucinations in Retrieval Augmented Generation. Experiments with a fine‑tuned Differentiable Search Index show that ToC‑based retrieval yields a low hallucination rate (<0.05%) and improves Recall@1 to 82.6% on the newly released SearchTome benchmark, outperforming baselines like BM25, DPR, and Mistral. The authors also release SearchTome, a diverse dataset of 18 books across six domains, to encourage further research in ToC‑based retrieval.
By Vineet Kumar, Meghanadh Pulivarthi, vishwajeet kumar, Jaydeep Sen, Riyaz Ahmad Bhat, Sachindra Joshi
arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.
By Alexander Apartsin, Yehudit Aperstein