REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs

The REAP system tackles the AKBC Shared Task 2026, aiming to build knowledge bases from language models without fine‑tuning and within a 32‑B parameter budget. It uses structured chain‑of‑thought reasoning, relation‑specific queries, and a reasoning‑based empty‑set gate to elicit knowledge, then directly extracts it into valid JSON arrays. Evaluated on the test set with the Mistral‑Small‑24B‑Instruct‑2501 model, REAP achieves a macro‑F1 of 0.62, notably high scores on countryLandBordersCountry (0.95), companyTradesAtStockExchange (0.73), and hasArea (0.77).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• llms
• rag
• fine-tuning