Doc2FRC introduces Fixed-Range Chunking (FRC), a dynamic programming method that partitions documents into chunks of a predefined length interval, ensuring consistent length distributions during training and inference. This approach reduces train-test length mismatch, mitigates n-gram repetition, and improves translation quality for 7B LLMs compared to direct Doc2Doc fine-tuning. Experiments on IWSLT2017 and a new 10-language test set, GlobVDoc, demonstrate that FRC outperforms existing document-level machine translation methods and enhances out-of-distribution translation performance.
By Xiaotian Wang, Youyuan Lin, Zhan Shen, Hitomi Yanaka
Intent-Driven Dynamic Chunking (IDC) segments documents by predicting user queries with a Large Language Model and then applying dynamic programming to find optimal chunk boundaries. This method outperforms traditional fixed-length or coherence-based segmentation on five out of six question-answering datasets, improving top-1 retrieval accuracy by 5% to 67% and reducing the number of chunks by 40–60% while maintaining 93–100% answer coverage. IDC demonstrates that aligning document structure with anticipated information needs can significantly boost retrieval performance for long and heterogeneous documents.
By Christos Koutsiaris
The paper presents an inference-only pipeline that extends the frozen NER model MahaNER‑BERT to document‑level prediction using overlapping sliding windows, eliminating the need for retraining or architectural changes. The approach is evaluated on six document‑level corpora derived from the MahaNER test set, employing two repetition strategies (Normal Repeat and Random Repeat) at three length levels and various window configurations. Results show the model maintains a macro F1‑score of up to 0.8902 with minimal variation, outperforming non‑windowed methods by avoiding boundary‑fragmentation errors and achieving more stable document‑level performance.
By Hariom Ingle, Ronit Ghode, Ishwari Gondkar, Jidnyasa Harad, Ravindra Murumkar, Raviraj Joshi
arXiv:2608. 10939v1 Announce Type: cross Abstract: Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet aggregate evaluation often hides large differences between high-resource and low-resource languages.
By Wajdi Ben Saad, Safa Madiouni
arXiv:2608.27658v1 Announce Type: new
Abstract: Subword tokenization hinders low-resource language processing by imposing frequency patterns from dominant languages onto script-sharing variants. Byte...
By Sanjeev Kumar, Atsuki Yamaguchi, Nikolaos Aletras
ChunkRank is an open‑source Python library that automatically determines chunk boundaries based on a target model’s tokenizer and context window, and then selects an answer from independently produced chunk candidates. It includes a registry of 90 models from 15 providers and six answer‑selection methods, and requires only three core dependencies. Experiments show that token‑exact budgeting is important across 11 languages, and that for several datasets no content‑based ranker outperforms simply taking the first non‑empty answer due to reader abstention on chunks lacking the answer.
By Amit Nautiyal, Ayush Bhatt, Gaurav Nautiyal
arXiv:2608. 06146v1 Announce Type: new Abstract: End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence.
By Hao Yu, Jiabo Zhan, Kang Liu, Linnan Zhao, Dongxu Yue, Rui Chen, Jinglin Wang, Chong Sun, Chen Li, Jing Lyu, Chun Yuan
End-to-end document parsers provide a unified interface, but serialize page layouts and regional contents into one autoregressive sequence. This formulation forces independent regions onto a decoding path whose length grows with the total content, whereas crop-based two-stage parsers expose region-level parallelism at the cost of repeated visual prefills and fragmented page context.
Efficient teamwork typically combines global coordination with parallel execution, a principle not yet fully reflected in unified Vision-Language Model (VLM)-based document parsers. Existing unified parsers process an entire page jointly but generate its output through a single token-by-token autoregressive trajectory, creating a sequential bottleneck that grows with document length.
arXiv:2607. 22644v1 Announce Type: new Abstract: Real-world document classification pipelines typically apply the same sequence of models to every incoming document, regardless of its complexity or type.
By Mohammed Yousif, Prabhjot Singh, Arjun Pankajakshan, Madhu Reddiboina
Sentence-level AI-generated text detection (S-AGTD) for hybrid documents, where humans and LLMs co-author one text, faces two gaps: existing methods classify each sentence in isolation, discarding inter-sentence dependencies, and existing benchmarks omit the newest generation of generators. We construct MOSAIC, a benchmark of 16,000 hybrid documents over PubMed and XSum, generated by DeepSeek-V3.
The paper compares Complement Naive Bayes (NB) with zero‑shot and few‑shot large language models (LLMs) across a wide range of model sizes and text classification tasks. NB outperforms LLMs when labeled data is available, achieving comparable accuracy to large LLMs while running thousands of samples per second on a CPU. In zero‑data sentiment settings, LLMs still dominate, but NB remains the best choice for resource‑constrained HPC practitioners, and the authors provide a Kubernetes Helm operator to automate model selection.
By Mohammad Firas Sada, Dmitry Mishin, John Graham, Seungmin Kim, Mahidhar Tatineni, Frank W\"urthwein