← Back to all news
arXiv Computation and Language By Elena Merdjanovska, Jonas Golde, Alan Akbik

Error-Type-Aware Loss Reweighting for Robust Named Entity Recognition with Noisy LLM Labels

Read the original on arXiv Computation and Language →

Large language models (LLMs) are increasingly used to annotate datasets for training smaller, task‑specialized models such as named entity recognition (NER). However, current fine‑tuning processes ignore the annotation noise introduced by LLMs, leading to degraded performance, and existing noise‑robust losses fail to handle the heterogeneous nature of NER noise (e.g., missing mentions vs. type errors). The authors propose error‑type‑aware loss reweighting, which applies separate reweighting rules for different erroneous token types, improving F1 scores by 0.8–2.0 percentage points on average and up to 4.6 points on Wikigold at 24.1% noise.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Sep 17

A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages

The paper introduces a scalable, multi-step framework designed to improve the quality of Named Entity Recognition (NER) annotations, particularly in low-resource languages. It employs a frequency-based iterative approach that combines self‑training with a dual‑threshold mechanism to increase inference confidence. Experiments on various NER datasets show notable performance gains over the original data, and the study also investigates the use of generative Large Language Models for NER tasks.

By Toqeer Ehsan, Thamar Solorio
llmsnlpbenchmarks
More like this →
arXiv Computation and Language
Oct 12

Assessment of Generative Named Entity Recognition in the Era of Large Language Models

The paper evaluates open‑source large language models (LLMs) for both flat and nested named entity recognition (NER) tasks, comparing them to traditional encoder‑based models and decoder‑based LLMs using in‑context learning. It shows that with parameter‑efficient fine‑tuning and structured output formats (e.g., inline brackets or XML), LLMs can match or exceed the performance of conventional NER systems, and that their NER abilities arise from instruction following rather than memorization. Additionally, fine‑tuning for NER minimally affects, and can even improve, the LLMs’ general capabilities on other benchmarks.

By Qi Zhan, Yile Wang, Hui Huang
llmsnlpfine-tuning
More like this →
arXiv Machine Learning
Sep 11

Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss

The paper proposes an information-weighted cross‑entropy loss that rescales token contributions using TF‑IDF statistics, thereby emphasizing semantically informative tokens and down‑weighting ubiquitous ones. Experiments on five decoder‑only language models (1.1B–13B parameters) show consistent reductions in memorized substring length while maintaining perplexity and downstream performance. The method is architecture‑agnostic, adds less than 3% computational overhead, and can be integrated into existing training pipelines.

By Zhijian Li, Stefan Larson, Kevin Leach
llmsfine-tuning
More like this →