Handwriting Trajectory Recovery with Diffusion Models

2026-07-03

AI Summary

This study addresses the challenging problem of converting offline handwritten images into online stroke trajectories—recovering the temporal sequence of pen-tip movements from static ink traces. To this end, it introduces, for the first time, an image-conditioned denoising diffusion model tailored to this task, trained and evaluated using Dynamic Time Warping (DTW/LDTW) and an Adaptive Intersection over Union (AIoU) metric. The proposed method substantially outperforms existing approaches such as PEN-Net and Cross-VAE on the CASIA-OLHWDB dataset, achieving notable improvements in both temporal consistency and shape fidelity. Moreover, it demonstrates strong generalization capabilities across character classes and even across writing systems, successfully transferring from Chinese characters to Latin alphabets.

Abstract

Recovering online pen trajectories from offline handwriting images, often referred to as handwriting trajectory recovery (stroke recovery), is an offline-to-online conversion task with applications in stroke-level editing and forensic analysis. We propose, to the best of our knowledge, the first diffusion-model-based framework for this task. Our method formulates trajectory recovery as image-conditioned generation and uses a denoising diffusion model to sample pen trajectories consistent with the observed ink trace. Through extensive quantitative evaluations on CASIA-OLHWDB (1.0-1.1), we verify that the proposed approach enables accurate recovery even for complex multi-stroke characters, substantially improving both temporal similarity (DTW/LDTW) and shape fidelity (AIoU) over representative prior methods such as PEN-Net and Cross-VAE. We further show that the model captures general stroke-order tendencies and generalizes to classes unseen during training, exemplified by cross-script transfer: a model trained on Chinese characters can recover reasonable stroke orders for Latin letters to some extent.

Problem

• handwriting trajectory recovery
• offline-to-online conversion
• stroke recovery
• pen trajectory
• handwriting analysis

Innovation

• diffusion models
• handwriting trajectory recovery
• offline-to-online conversion
• stroke-order generalization
• cross-script transfer