HandwritingAgent: Language-Driven Handwriting Synthesis in Scalable Vector Space

2026-06-17

AI Summary

Existing approaches struggle to naturally emulate human handwriting due to the challenge of dynamically generating stroke sequences with high variability in shape, pen pressure, and trajectory, as well as the lack of flexible style control via natural language. This work proposes the first language-driven handwriting synthesis agent that requires no style-specific training, autoregressively generating stroke sequences in SVG space over a discrete grid canvas and enabling joint control through textual instructions and reference images. By integrating large reasoning models with geometric analysis, the method efficiently and robustly synthesizes complex content—including multilingual text and mathematical expressions—achieving performance on par with or superior to state-of-the-art methods in handwriting imitation, recognition, and synthesis tasks, while significantly enhancing controllability, efficiency, and generalization.

Abstract

Teaching machines to emulate natural handwriting styles remains an open challenge, as it requires synthesizing stroke sequences that dynamically vary in shape, texture, pressure and script - not only across individuals, but also within a single person's handwriting. Attempts at this challenge have largely explored deep learning methods in both online and offline settings. However, these approaches are often constrained by style-specific architectural choices, heavy reliance on large datasets, high compute costs, and a lack of flexible control over writing styles through natural language. To this end, we introduce HandwritingAgent, a language-driven agent that can synthesize natural handwriting sequences directly in Scalable Vector Graphics (SVG) format with no need for style-specific training. The agent leverages a large reasoning model to geometrically analyse and autoregressively generate target handwritten glyphs as stroke sequences in a discrete grid canvas environment. Generation is conditioned on texts provided in either conversational or non-conversational mode, along with a reference handwriting-style image. Experiments on diverse handwriting tasks spanning imitation, recognition, multi-lingual handwriting synthesis, and generation of complex handwritten maths and science expressions indicate substantial improvement in performance, with HandwritingAgent matching or surpassing state-of-the-art generative handwriting models, while providing a more efficient, controllable, and generalizable synthesis method.

Problem

• handwriting synthesis
• natural handwriting styles
• language-driven control
• stroke sequences
• style variation

Innovation

• language-driven handwriting synthesis
• Scalable Vector Graphics (SVG)
• large reasoning model
• style-controllable generation
• autoregressive stroke modeling