Parquet Content-Defined Chunking

Building a Data Lakehouse with DuckDB and DuckLake

Starting with a local Parquet file, then joining it to data stored in the cloud The post Building a Data Lakehouse with DuckDB and DuckLake appeared first on Towards Data Science.

SemChunk-C: Semantic Segmentation for C Code

arXiv:2606. 23697v1 Announce Type: cross Abstract: Semantic segmentation of code written in a C-family language remains a challenging problem, due to the language's complex syntax, macro expansion, and irregular structural patterns.

Configurable Semantic Chunking for Biomedical Information Extraction in Retrieval-Augmented Generation

The paper introduces a configurable semantic chunking framework for biomedical information extraction in retrieval‑augmented generation systems. It replaces the fixed‑size chunking stage of BioMedRAG with entity‑preserving windows, trigger‑centered chunking, proposition‑first extraction, tiered trigger prioritization, and hierarchical relation resolution, while keeping the rest of the pipeline unchanged. Experiments on relation extraction benchmarks (GM‑CIHT, DDI, ChemProt) and adverse event classification (ADE) show that the hybrid configuration boosts performance on datasets with explicit relation cues, achieving 82.6% F1 on GM‑CIHT compared to 74.2% with the baseline.

Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking

The paper evaluates how different chunking strategies—fixed-size, recursive paragraph-first, and content-defined—affect the cost of updating embeddings when Wikipedia documents are revised. Using 120 pairs of Chinese and English revisions, the authors find that fixed-size chunking requires the most recomputation (57‑68 % of the workload), while recursive and content-defined chunking need less (38‑56 % and 43‑54 % respectively). Despite these differences, evidence recall under a 2048‑token retrieval budget varies by less than two percentage points across methods and languages.

UnWeaving the knots of GraphRAG -- turns out VectorRAG is almost enough

arXiv:2603. 29875v3 Announce Type: replace-cross Abstract: One of the key problems in Retrieval-augmented generation (RAG) systems is that chunk-based retrieval pipelines represent the source chunks as atomic objects, mixing the information contained within such a chunk into a single vector.

From Files to Chunks: Improving HF Storage Efficiency