Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation

The study investigates how document segmentation and chunk representation affect retrieval-augmented generation (RAG) for chemistry texts. Using the ChemQuests corpus, the authors benchmark 41 embedding models and evaluate them across five chunking strategies, seven chunk sizes, and various overlap settings. They find that embedding choice has the largest impact, with models like E5, BGE, and Nomic performing best, and recommend medium-to-large chunks with fixed-token, recursive-token, or hierarchical-section chunking and low overlap for effective chemistry-aware RAG.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• rag
• computer-vision
• nlp
• benchmarks