Does My Embedding Reflect That $A = B$? Evaluating Mathematical Equivalence in Embedding Models

2026-06-22

AI Summary

Existing embedding models struggle to recognize mathematically equivalent statements that differ significantly in linguistic expression, hindering cross-domain association and retrieval of mathematical knowledge. To address this limitation, this work presents the first systematic evaluation of embedding models’ capacity to capture mathematical equivalence, introduces MELD—the first large-scale benchmark dataset for this purpose—and proposes a contrastive learning framework specifically designed for mathematical semantic alignment. The framework aligns informal natural language statements with their diverse formalized counterparts. Experimental results demonstrate that the proposed approach substantially outperforms state-of-the-art models on both informal-to-formal retrieval and purely natural language tasks within MELD, effectively enhancing the model’s understanding of underlying mathematical semantics rather than superficial lexical features.

Abstract

Because mathematics is highly abstract, a single statement can take very different forms depending on what subfield it is framed in. There are many examples where breakthroughs occurred after researchers discovered that a question had already been answered in a different field. At the same time, the growth of new resources related to formalization has increased the need for tools that enable efficient and reliable navigation between mathematical 'languages' (e.g., from Lean to natural language). In this paper, we investigate whether current embedding models capture mathematical equivalence. To do this, we introduce the Mathematically Equivalent but Lexically Different Pairs (MELD) Dataset, a collection of mathematically equivalent statements that are expressed in very different language. We show that current state-of-the-art embedding models tend to group statements by the terminology used to make them instead of the underlying math. Motivated by this, we propose a contrastive approach to learning embeddings of mathematical text that focuses on aligning informal statements with different formalizations. Our experiments demonstrate that this leads to improvements not only on informal-formal retrieval tasks but also on MELD, which only contains natural language statements.

Problem

• mathematical equivalence
• embedding models
• natural language
• formalization
• lexical variation

Innovation

• mathematical equivalence
• embedding models
• contrastive learning
• MELD dataset
• formalization