arXiv:2606. 26246v1 Announce Type: cross Abstract: Lacuna is a research map for machine learning that uses LLMs to turn papers and scholarly metadata into markdown summaries, concept elements, research directions, and research proposals.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
SHELF is a Python system that creates synthetic, controlled benchmark data for evaluating large language models on bibliographic tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. It generates 62,899 model-written documents based on Library of Congress vocabularies and compares methods like TF, TF‑IDF, BM25, popular encoders, and zero‑shot decoders, reporting performance metrics such as 0.8887 for subject classification and 0.2605 for genre‑form classification. The tool also allows independent variation of bibliographic facets and can produce unseen documents beyond a model’s training cutoff, with results indicating that model rankings transfer more reliably than absolute scores when compared to other benchmarks.
The paper reports the ABAI submission to COLIEE 2026 Task 1, a case law retrieval challenge that suppresses cited passages, and details a four‑stage retrieval pipeline: multi‑view BM25 with reciprocal rank fusion, neural reranking, graph‑based features via a graph attention network, and a LightGBM meta‑learner over 34 features. The best run achieved an F1 score of 0.177 on the official test set, compared to a cross‑validated 0.311, and the authors attribute the gap to a recall ceiling, temporal distribution shift, and threshold miscalibration. A controlled post‑hoc study examined the impact of threshold transfer, decision quality across time, and query similarity, and identified specific remedies—such as BM25 length‑normalisation tuning, event‑triple views, and dense fusion—that improved recall, while other interventions had no effect.
SHELF is a Python system that creates controlled benchmark data and evaluation tasks for libraries and archives, using labelled taxonomies, writing specifications, and a generation budget. It generates 62,899 model-written documents based on Library of Congress vocabularies and supports tasks such as classification, clustering, retrieval, pair classification, and instruction retrieval. The release compares various methods—including TF, TF-IDF, BM25, popular encoders, and zero-shot decoders—showing that sparse methods remain competitive on classification and that SHELF can vary bibliographic facets independently while generating new, verifiably unseen documents.
OpenResearcher is a fully open, reproducible pipeline for generating long‑horizon deep research trajectories that interleave search, evidence aggregation, and multi‑step reasoning. It decouples corpus bootstrapping from trajectory synthesis and runs the search‑and‑browse loop offline using three browser primitives over a 15M‑document corpus. Using GPT‑OSS‑120B as a teacher, the pipeline produced over 97K trajectories, enabling a 30B‑A3B model to achieve 54.8% accuracy on BrowseComp‑Plus and providing insights into pipeline design through controlled analysis.
T-Search is an open-weight agentic retriever designed for hard multi-step search tasks. It performs bounded multi-round searches over a fixed corpus, returning ranked evidence chunks with brief justifications, while allowing the answer generation component to be swapped without retraining. Trained on synthetic search tasks and evaluated on seven English and Russian benchmarks, it achieves 56.0 Recall@10 with one rollout and 61.3 with three fused rollouts, surpassing larger open models.
The paper introduces a GPU‑optimized retrieval framework for LinkedIn’s semantic search, partitioning embeddings into eight category‑supervised segments and applying a min/median aggregation rule aligned with the existing relevance policy. A lightweight Stage‑1 scorer generates high‑recall candidates, while a two‑stage GPU architecture—FP8 coarse ranking followed by FP16 re‑ranking—boosts throughput and recall, achieving 99.6‑99.8% of full‑FP16 recall at over 500 QPS per shard. In A/B testing, the system raises exploratory‑query Precision@10 from 63.7% to 79.0% and navigational Precision@1 from 65.5% to 74.7%, with human evaluation confirming the improvement.