SHELF is a Python system that creates synthetic, controlled benchmark data for evaluating large language models on bibliographic tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. It generates 62,899 model-written documents based on Library of Congress vocabularies and compares methods like TF, TF‑IDF, BM25, popular encoders, and zero‑shot decoders, reporting performance metrics such as 0.8887 for subject classification and 0.2605 for genre‑form classification. The tool also allows independent variation of bibliographic facets and can produce unseen documents beyond a model’s training cutoff, with results indicating that model rankings transfer more reliably than absolute scores when compared to other benchmarks.
By Michael J. Bommarito II
The paper reports the ABAI submission to COLIEE 2026 Task 1, a case law retrieval challenge that suppresses cited passages, and details a four‑stage retrieval pipeline: multi‑view BM25 with reciprocal rank fusion, neural reranking, graph‑based features via a graph attention network, and a LightGBM meta‑learner over 34 features. The best run achieved an F1 score of 0.177 on the official test set, compared to a cross‑validated 0.311, and the authors attribute the gap to a recall ceiling, temporal distribution shift, and threshold miscalibration. A controlled post‑hoc study examined the impact of threshold transfer, decision quality across time, and query similarity, and identified specific remedies—such as BM25 length‑normalisation tuning, event‑triple views, and dense fusion—that improved recall, while other interventions had no effect.
By Minhan Cho, Soyoung Park, Daejin Choi, Jinyoung Han
SHELF is a Python system that creates controlled benchmark data and evaluation tasks for libraries and archives, using labelled taxonomies, writing specifications, and a generation budget. It generates 62,899 model-written documents based on Library of Congress vocabularies and supports tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. The release compares various methods—including TF, TF-IDF, BM25, popular encoders, and zero-shot decoders—showing that sparse methods remain competitive on classification and that SHELF can vary bibliographic facets independently while generating new, verifiably unseen documents.
OpenResearcher is a fully open, reproducible pipeline for generating long‑horizon deep research trajectories that interleave search, evidence aggregation, and multi‑step reasoning. It decouples corpus bootstrapping from trajectory synthesis and runs the search‑and‑browse loop offline using three browser primitives over a 15M‑document corpus. Using GPT‑OSS‑120B as a teacher, the pipeline produced over 97K trajectories, enabling a 30B‑A3B model to achieve 54.8% accuracy on BrowseComp‑Plus and providing insights into pipeline design through controlled analysis.
By Zhuofeng Li, Dongfu Jiang, Xueguang Ma, Haoxiang Zhang, Ping Nie, Yuyu Zhang, Kai Zou, Jianwen Xie, Yu Zhang, Wenhu Chen
T-Search is an open-weight agentic retriever designed for hard multi-step search tasks. It performs bounded multi-round searches over a fixed corpus, returning ranked evidence chunks with brief justifications, while allowing the answer generation component to be swapped without retraining. Trained on synthetic search tasks and evaluated on seven English and Russian benchmarks, it achieves 56.0 Recall@10 with one rollout and 61.3 with three fused rollouts, surpassing larger open models.
The paper introduces a GPU‑optimized retrieval framework for LinkedIn’s semantic search, partitioning embeddings into eight category‑supervised segments and applying a min/median aggregation rule aligned with the existing relevance policy. A lightweight Stage‑1 scorer generates high‑recall candidates, while a two‑stage GPU architecture—FP8 coarse ranking followed by FP16 re‑ranking—boosts throughput and recall, achieving 99.6‑99.8% of full‑FP16 recall at over 500 QPS per shard. In A/B testing, the system raises exploratory‑query Precision@10 from 63.7% to 79.0% and navigational Precision@1 from 65.5% to 74.7%, with human evaluation confirming the improvement.
By Dhritiman Das, Chujie Zheng, Ronak Kaoshik, Pratik Dixit, Vishal Shah, Yanbo Li, Jiahao Xu, Manika Agarwal, Chinmay Naik, Lingyu Zhang, Chetan Bhole, Chirag Bhanuprasad Mehta, Meng Zheng, Puneet Singh Ahluwalia, Shirisha Singh, Ping Jin, Manas Apte, Gokulraj Mohanasundaram, Tugrul Bingol, Raghavan Muthuregunathan, Fedor Borisyuk
RIT‑RAG is a retrieval‑augmented generation framework that combines content retrieval with structural navigation of documents. It builds a tree for each document from its table of contents or sitemap, then uses retrieved chunk positions to induce manageable sub‑trees that an LLM agent can navigate, read selectively, and reformulate queries. Experiments on financial, scientific, and customer‑support benchmarks show RIT‑RAG outperforms vanilla, graph‑based, and agentic baselines, achieving up to 11.4‑point accuracy gains on the new EntQABench of 2.84 million technical‑documentation webpages.
By Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
arXiv:2607. 26670v1 Announce Type: cross Abstract: The rapid growth of scholarly literature has made identifying relevant publications increasingly difficult, and conventional search systems still depend heavily on manually formulated queries and effortful manual inspection.
By Eleni Adamidi, Serafeim Chatzopoulos, Thanasis Vergoulis
arXiv:2609.40340v1 Announce Type: new
Abstract: Evolutionary search with large language models (LLMs) can stall when progress requires external knowledge the model lacks. Supplying relevant documents...
By Young-Jun Lee, Jinheon Baek, Soyeong Jeong, Minki Kang, Seungyeon Jwa, Jonghyun Choi, Seungho Han, Dongyeop Kang
Evolutionary search with large language models (LLMs) can stall when progress requires external knowledge the model lacks. Supplying relevant documents helps, but simply adding web search tool can kee...
arXiv:2610.07132v1 Announce Type: cross
Abstract: Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful rea...
By Berke Arda, Ahmetcan Yavuz, Paul Gerry, Sebastian Lobentanzer, Nobin Sarwar, Joan Giner-Miguelez, Kongtao Chen, Luyao Zhang, Mrinmaya Sachan, Mubashara Akhtar
Benchmark Radar is a living database and search engine that aggregates AI benchmark papers, datasets, code, and score histories. It automatically discovers new benchmark resources from 37 sources, maintains a catalog of 1,283 records with 12,916 numeric observations, and provides tools such as a web dashboard, CLI, and downloadable evidence for researchers. The system also offers visualizations like a Pareto frontier and trend views to help users assess benchmark saturation and adoption.
By Koutian Wu, Junjie Zhou, Ergan Shang, Jiayu Wang, Pengqian Han, Junkai Wang, Wanghan Xu