MASA: LLM-Driven Multi-Agent Systems for Autoformalization

2025-10-10

AI Summary

This work addresses the problem of automated translation from natural language to formal representations. We propose a large language model (LLM)-driven modular multi-agent system that decomposes the task into coordinated subproblems—natural language proposition parsing, formal translation, and theorem prover verification—via agent specialization and dynamic tool integration, enabling end-to-end formalization. Our key contribution lies in deeply coupling LLMs’ semantic understanding capabilities with external formal reasoning tools, thereby establishing a scalable, maintainable, and extensible collaborative framework. Experiments on real-world mathematical definitions and formal mathematics benchmarks—including MiniF2F and HOL-Test—demonstrate significant improvements: +18.3% absolute gain in formalization accuracy and enhanced inference reliability, while also improving system flexibility and cross-task generalization capability.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Autoformalization serves a crucial role in connecting natural language and formal reasoning. This paper presents MASA, a novel framework for building multi-agent systems for autoformalization driven by Large Language Models (LLMs). MASA leverages collaborative agents to convert natural language statements into their formal representations. The architecture of MASA is designed with a strong emphasis on modularity, flexibility, and extensibility, allowing seamless integration of new agents and tools to adapt to a fast-evolving field. We showcase the effectiveness of MASA through use cases on real-world mathematical definitions and experiments on formal mathematics datasets. This work highlights the potential of multi-agent systems powered by the interaction of LLMs and theorem provers in enhancing the efficiency and reliability of autoformalization, providing valuable insights and support for researchers and practitioners in the field.

Problem

• Converting natural language to formal representations using LLMs
• Building modular multi-agent systems for autoformalization tasks
• Enhancing reliability of formal reasoning through collaborative agents

Innovation

• Multi-agent system for autoformalization using LLMs
• Modular architecture integrating agents and tools
• LLM-driven agents collaborating with theorem provers