arXiv AI
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Aug 21
EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation
arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.
arXiv Machine Learning
2d ago
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
The paper proposes an information-weighted cross‑entropy loss that rescales token contributions using TF‑IDF statistics, thereby emphasizing semantically informative tokens and down‑weighting ubiquitous ones. Experiments on five decoder‑only language models (1.1B–13B parameters) show consistent reductions in memorized substring length while maintaining perplexity and downstream performance. The method is architecture‑agnostic, adds less than 3% computational overhead, and can be integrated into existing training pipelines.
arXiv AI
Jun 2
What Do LLMs Know About Alzheimer's Disease? Multi-loss Fine-Tuning and Probing for AD Detection
arXiv:2602. 11177v2 Announce Type: replace-cross Abstract: Reliable early detection of Alzheimer's disease (AD) is challenging, particularly due to the limited availability of labeled data.
arXiv Computation and Language
Sep 2
DECSELFMASK: Leveraging Unlabeled Text via Self-Relevance-Guided Masking for Decoder-Only Classification
DECSELFMASK is a decoder‑only classification method that uses unlabeled clinical text to improve performance. It creates self‑supervised training examples by masking portions of the text identified as relevant through relevance attribution, then trains the model to reconstruct the masked tokens via next‑token prediction. Experiments on 136 tasks from 1.9 M Italian hospital notes show consistent gains across five models, outperforming base models (+9.1 Macro F1), continual pretraining (+6.3), and synthetic label generation (+12.5).
Hugging Face Trending Papers
3d ago
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
Large language models are typically trained under uniform token weighting, which allows frequent and low-information tokens to dominate learning and can increase the tendency to memorize surface-level...
arXiv AI
Jun 24
Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Models
arXiv:2606. 24841v1 Announce Type: new Abstract: Prompt-based learning has emerged as a dominant paradigm in natural language processing.