Int2Int: a framework for mathematics with transformers

2025-02-22

AI Summary

This work addresses the inability of large language models to model integer structures in pure mathematics—particularly number theory. We propose the first open-source, end-to-end Transformer framework specifically designed for integer-centric mathematical problems. Methodologically, we introduce a math-aware encoder supporting hybrid symbolic-numerical representations—including integer sequences, modular forms, and L-functions—alongside a domain-specific data preprocessing pipeline, differentiable decoding, and integrated Jupyter-based visualization. Our contributions are threefold: (1) the first deep integration of deep learning with structural number-theoretic objects; (2) a complete, MIT-licensed PyTorch implementation—including training/evaluation scripts and reproducible benchmarks; and (3) a substantial reduction in barriers to AI-for-math research, enabling systematic deployment of foundation models in fundamental mathematics.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper documents Int2Int, an open source code base for using transformers on problems of mathematical research, with a focus on number theory and other problems involving integers. Int2Int is a complete PyTorch implementation of a transformer architecture, together with training and evaluation loops, and classes and functions to represent, generate and decode common mathematical objects. Ancillary code for data preparation, and Jupyter Notebooks for visualizing experimental results are also provided. This document presents the main features of Int2Int, serves as its user manual, and provides guidelines on how to extend it. Int2Int is released under the MIT licence, at https://github.com/FacebookResearch/Int2Int.

Problem

• transformer framework for mathematics
• focus on number theory
• PyTorch implementation and tools

Innovation

• Transformers for mathematical research
• PyTorch implementation of transformer
• Open source mathematical framework