arXiv AI
6d ago

inkGround: Benchmarking Fine-grained Visual Grounding Capabilities of Vision-Language Models for Cursive Handwriting in the Wild

inkGround is a new benchmark for evaluating how well Vision‑Language Models can recover the fine‑grained stroke trajectories of cursive handwriting from camera‑captured page images. It contains 537 full‑page images with 18.7k words and 163k strokes, paired with pseudo‑trajectory references generated by an offline‑to‑online model and verified by humans. The benchmark casts the task as text‑conditioned point grounding, scoring models on trajectory fidelity, readability, and style similarity, and compares both open‑source and closed‑source VLMs, showing that closed‑source models like Gemini‑3.1‑Pro perform best.

By Sidra Hanif
arXiv AI
Oct 16

InkSight: Offline-to-Online Handwriting Conversion by Teaching Vision-Language Models to Read and Write

InkSight is a system that converts offline handwritten notes captured in photos into digital ink, enabling seamless transition from pen-and-paper to editable, vectorized formats. Unlike prior methods that rely heavily on geometric image properties, InkSight integrates reading and writing priors, allowing it to learn from limited paired data. The model successfully derenders handwritten text in diverse photos and even simple sketches, with human evaluations showing 87% of outputs as valid tracings and 67% resembling human pen trajectories.

By Blagoj Mitrevski, Arina Rak, Julian Schnitzler, Chengkun Li, Andrii Maksai, Jesse Berent, Claudiu Musat
arXiv Machine Learning
Jul 14

Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition

arXiv:2603. 16883v2 Announce Type: replace-cross Abstract: Inertial measurement unit-based online handwriting recognition enables the recognition of input signals collected across different writing surfaces but remains challenged by uneven character distributions and inter-writer variability.

By Jindong Li, Dario Zanca, Vincent Christlein, Tim Hamann, Jens Barth, Peter K\"ampf, Bj\"orn Eskofier