Natural language processing

Classical and neural NLP: translation, question answering, tokenization and the evaluation of language understanding.

2,564 stories · RSS feed

arXiv Machine Learning
5d ago

Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers

The paper investigates how the choice of graph tokenization affects transformer expressivity. It analyzes three tokenization families—spectral, random‑walk, and adjacency—showing that each induces different depth requirements and that some tokenizations are inherently lossy or ill‑conditioned for certain tasks. The authors prove lower bounds and impossibility results for converting between tokenizations and validate these findings with experiments on synthetic and real‑world data.

By Maya Bechler-Speicher, Gilad Yehudai, Gil Harari, Clayton Sanford, Amir Globerson, Joan Bruna
Sebastian Raschka
5d ago

Language Models for Text Classification: From Bag-of-Words to Jev

The article titled "Language Models for Text Classification: From Bag-of-Words to Jev" offers a visual guide that explores various neural network architectures—including RNNs, CNNs, and Transformers—alongside calibration techniques. It includes hands‑on experiments that compare the accuracy and efficiency of these models for text classification tasks.

By Sebastian Raschka, PhD