This paper introduces Sentence Splitter, a self-supervised framework built upon a T5-based encoder--decoder architecture for uncovering the latent factual structure of natural language sentences. The proposed method identifies the semantic boundary between a descriptive prefix (head) and its factual completion (tail) by formulating sentence splitting as a discrete segmentation problem, where a sentence of length $N$ admits $N$ possible split points but only one recovers the intended head--tail structure.
arXiv:2607. 19845v1 Announce Type: cross Abstract: This paper introduces Sentence Splitter, a self-supervised framework built upon a T5-based encoder--decoder architecture for uncovering the latent factual structure of natural language sentences.
arXiv:2608.30627v1 Announce Type: new Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...
arXiv:2609.15338v1 Announce Type: cross Abstract: Large Language Models (LLMs) primarily perform inference at the token level, resulting in substantial memory overhead and compromised computational e...
arXiv:2607. 19354v1 Announce Type: new Abstract: Spreadsheet applications are used by hundreds of millions worldwide, yet writing formulas remains a significant barrier.
arXiv:2606. 15741v1 Announce Type: cross Abstract: Narrative question answering (NQA) is a challenging task in natural language processing that requires models to understand long textual contexts, capture relationships across events, and generate coherent responses.
The paper proposes a self‑supervised framework that trains language models to predict concepts—sets of semantically equivalent tokens—rather than single tokens. This approach improves alignment with human similarity judgments, boosts performance on classification, clustering, and reranking tasks, and yields comparable or stronger downstream reasoning while lowering perplexity on semantically meaningful words and only slightly increasing overall perplexity.
arXiv:2603. 05299v2 Announce Type: replace-cross Abstract: Large language models show that simple autoregressive training can yield scalable and coherent generation, but extending this paradigm to speech remains challenging due to the entanglement of semantic and acoustic information.
arXiv:2601. 15037v2 Announce Type: replace-cross Abstract: Open-domain Relational Triplet Extraction (ORTE) aims to mine structured knowledge without predefined relation schemas.
arXiv:2608. 19529v1 Announce Type: cross Abstract: Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language.
The paper introduces Retrieval-Augmented Decoding (RAD), a decoding-time method that improves the truthfulness of large language models without retraining. RAD uses a small reference set of up to ten annotated examples to build a grounding space of context embeddings and next-token logits, which it retrieves and aggregates during inference to shape the model’s output. Experiments on four open-ended generation benchmarks and four different LLMs show that RAD consistently outperforms strong baselines and generalizes well across tasks.
DECSELFMASK is a decoder‑only classification method that uses unlabeled clinical text to improve performance. It creates self‑supervised training examples by masking portions of the text identified as relevant through relevance attribution, then trains the model to reconstruct the masked tokens via next‑token prediction. Experiments on 136 tasks from 1.9 M Italian hospital notes show consistent gains across five models, outperforming base models (+9.1 Macro F1), continual pretraining (+6.3), and synthetic label generation (+12.5).
Chain-of-thought (CoT) reasoning has emerged as an effective approach for activating latent reasoning capabilities in large language models. However, most existing CoT methods use reasoning chains mainly as inference-time prompts, while the generated reasoning traces are rarely reused as semi-supervised learning signals.