arXiv AI

HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixtures

arXiv:2607. 02266v1 Announce Type: cross Abstract: Most data-mixing methods assume the corpus has already been partitioned into groups, and the choice of those groups determines what a mixer can express.

arXiv Machine Learning
Jul 30

The Advantage of Fine-Grained Training

arXiv:2509. 05130v2 Announce Type: replace Abstract: In classification problems, models are trained to predict a class label based on the input data features.

By Davide Pirovano, Federico Milanesio, Michele Caselle, Piero Fariselli, Matteo Osella
arXiv Computation and Language
Aug 31

Nested Byte-Level Vocabularies Are Cheap to Deploy and Expensive to Share: A Pre-Registered Negative Result

The paper demonstrates that a byte‑level BPE tokenizer can be sliced to create multiple vocabulary sizes from a single trained model, preserving exact logits while reducing deployed weights by 66%. Experiments on 30 models show that while sliced models match the full model numerically, they underperform fixed‑cap specialists by a few percentage points in bits‑per‑byte. Multi‑cap training improves robustness to typographical noise, suggesting benefits from training across multiple granularities rather than from control tokens alone.

By Christos Koutsiaris
arXiv AI
Aug 25

Hyperbolic Hierarchical Clustering for Visual Representation Learning

The paper introduces ClusterMixer, a token mixer based on hierarchical clustering in hyperbolic space, designed to be transparent and interpretable. It forms the core of a new vision backbone called HCFormer, which incorporates multiple clustering strategies to maintain strong performance. Experiments show HCFormer surpasses existing backbones on tasks such as image classification, object detection, instance segmentation, and semantic segmentation.

By Jianan Wei, Guikun Chen, Zhiyuan Weng, Chunchao Guo, Yujia Wang, Wenguan Wang
arXiv Computation and Language
Aug 25

LakeHopper: Knowledge-Aware Adaptation of Column Type Annotators across Data Lakes

LakeHopper is a method for adapting column type annotators (CTA) from one data lake to another by treating cross‑lake adaptation as a knowledge‑management problem. It decomposes the source annotator’s knowledge into source‑specific, shared, and target‑specific parts, and then uses three mechanisms—label‑set realignment, LLM‑verified gap discovery, and cluster‑based propagation with rehearsal fine‑tuning—to adapt the annotator under a limited annotation budget. The approach achieves up to a 71.4% relative macro‑F1 improvement over three PLM backbones, reaches near‑full data quality with less than 6% of target labels, and trains 27–131 times faster than fine‑tuned table LLMs.

By Yushi Sun, Xujia Li, Nan Tang, Quanqing Xu, Chuanhui Yang, Lei Chen