arXiv AI By You Zuo (ALMAnaCH), Kim Gerdes (LISN), Eric Villemonte de La Clergerie (ALMAnaCH), Beno\^it Sagot (ALMAnaCH)

Patent Representation Learning via Self-supervision

Read the original on arXiv AI →

arXiv:2511. 10657v2 Announce Type: replace-cross Abstract: We study self-supervised patent representation learning with contrastive objectives.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 2

DECSELFMASK: Leveraging Unlabeled Text via Self-Relevance-Guided Masking for Decoder-Only Classification

DECSELFMASK is a decoder‑only classification method that uses unlabeled clinical text to improve performance. It creates self‑supervised training examples by masking portions of the text identified as relevant through relevance attribution, then trains the model to reconstruct the masked tokens via next‑token prediction. Experiments on 136 tasks from 1.9 M Italian hospital notes show consistent gains across five models, outperforming base models (+9.1 Macro F1), continual pretraining (+6.3), and synthetic label generation (+12.5).

By Pietro Ferrazzi, Matteo Merler, Giovanni Bonetta, Alberto Lavelli, Bernardo Magnini
arXiv AI
Aug 19

Sparse Coverage: Semantic Center Representations for Patent Prior-Art Retrieval

Sparse Coverage is an unsupervised semantic retrieval framework designed for patent prior‑art search. It maps local span embeddings to a sparse vocabulary of embedding‑space centers chosen via a coverage‑oriented k‑center objective, allowing spans to activate nearby centers and produce sparse representations that work with inverted‑index retrieval. Experiments on CLEF‑IP 2013 demonstrate that Sparse Coverage matches or surpasses dense patent encoders in document‑level recall while remaining competitive at the passage level, making it an effective first‑stage retrieval approach for patent search.

By You Zuo (ALMAnaCH), Kim Gerdes (LISN, Qatent, STL), \'Eric de la Clergerie (ALMAnaCH), Beno\^it Sagot (ALMAnaCH)