DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation

2026-06-18

AI Summary

This work addresses the challenges of handwritten mathematical expression generation, which involve complex two-dimensional layouts and long-range structural dependencies. Existing approaches rely on costly positional annotations, limiting their scalability. To overcome this, we propose DiffMath—a framework that eliminates the need for explicit spatial supervision by encoding LaTeX hierarchical structures into symbol-relation-depth ([S, R, D]) triplet sequences. DiffMath integrates a relation-aware abstract syntax tree (RelAST), a structure-aware MathVAE, and a latent-space diffusion Transformer (MathDiT) with AdaLN conditioning to generate mathematically coherent and visually high-quality expressions. Our method significantly outperforms current state-of-the-art models and demonstrates practical utility by effectively enhancing the recognition accuracy of downstream optical character recognition (OCR) systems through synthetic data augmentation.

Abstract

Handwritten Mathematical Expression Generation (HMEG) is challenging due to the complex two-dimensional layouts and long-range structural dependencies of mathematical expressions. Existing methods typically rely on explicit spatial supervision, such as symbol-level bounding boxes, which incurs high annotation costs and limits scalability. In this work, we propose DiffMath, a symbol- and graph-aware latent diffusion framework that leverages the hierarchical structure inherent in LaTeX as a structural prior, eliminating the need for positional supervision. First, we design a Relational Abstract Syntax Tree (RelAST), a generation-oriented representation that distills MathML trees into compact triplet sequences [S, R, D], where each token directly encodes a symbol identity, spatial relation, or nesting depth. Second, we introduce MathVAE, which learns structure-preserving latent representations through symbol-aware and relation-aware perceptual regularization, ensuring that the latent space captures both character semantics and spatial topology. Third, MathDiT performs conditional denoising in this structured latent space, further guided by a global symbol-count prior via Adaptive Layer Normalization (AdaLN) to improve structural coherence. Experiments show that DiffMath produces structurally consistent handwritten expressions, achieves superior performance over existing methods, and improves the accuracy of downstream OCR models through synthetic data augmentation.

Problem

• Handwritten Mathematical Expression Generation
• structural dependencies
• spatial supervision
• annotation cost
• scalability

Innovation

• latent diffusion
• structural prior
• Relational Abstract Syntax Tree
• symbol-aware generation
• handwritten mathematical expression