Towards a Densing Law for User Representation Learning at Billion-Scale Capacity

The paper introduces a User Behavioral Densing Law that quantifies how the minimum sufficient tokenization capacity scales with data size in user representation learning. A pilot study on a billion‑scale Alipay dataset shows raw data scaling bottlenecks and the benefits of tokenization, while theoretical analysis and experiments reveal an approximately linear relationship between the logarithms of tokenization capacity and input data size. Using this law, the authors develop ALGN, an adaptive variable‑length tokenization method that outperforms existing baselines across diverse data sources and downstream tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• nlp