AI Assistants to Enhance and Exploit the PETSc Knowledge Base

2025-06-25

AI Summary

Decades of informal knowledge—scattered across source code, mailing lists, issue trackers, and developer discussions—within the PETSc library remain fragmented and underutilized, severely hindering user support and documentation maintenance. Method: We propose a knowledge-centric AI framework for HPC libraries, integrating retrieval-augmented generation (RAG), semantic re-ranking, and a domain-specific toolchain to formalize and dynamically leverage informal knowledge. Our system constructs a unified embedding index from heterogeneous sources, synergistically combines large language models with automated documentation update mechanisms, and supports question answering, development assistance, and technical documentation generation. Contribution/Results: Evaluated on the Argonne Leadership Computing Facility’s supercomputing platform, the framework demonstrates high accuracy, scalability, and engineering practicality in the context of scalable Krylov solvers. It significantly improves knowledge reuse efficiency and developer productivity in scientific computing software ecosystems.

Technology Category

• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Generative AI, especially through large language models (LLMs), is transforming how technical knowledge can be accessed, reused, and extended. PETSc, a widely used numerical library for high-performance scientific computing, has accumulated a rich but fragmented knowledge base over its three decades of development, spanning source code, documentation, mailing lists, GitLab issues, Discord conversations, technical papers, and more. Much of this knowledge remains informal and inaccessible to users and new developers. To activate and utilize this knowledge base more effectively, the PETSc team has begun building an LLM-powered system that combines PETSc content with custom LLM tools -- including retrieval-augmented generation (RAG), reranking algorithms, and chatbots -- to assist users, support developers, and propose updates to formal documentation. This paper presents initial experiences designing and evaluating these tools, focusing on system architecture, using RAG and reranking for PETSc-specific information, evaluation methodologies for various LLMs and embedding models, and user interface design. Leveraging the Argonne Leadership Computing Facility resources, we analyze how LLM responses can enhance the development and use of numerical software, with an initial focus on scalable Krylov solvers. Our goal is to establish an extensible framework for knowledge-centered AI in scientific software, enabling scalable support, enriched documentation, and enhanced workflows for research and development. We conclude by outlining directions for expanding this system into a robust, evolving platform that advances software ecosystems to accelerate scientific discovery.

Problem

• Enhance access to fragmented PETSc knowledge base
• Develop LLM tools for user and developer support
• Create extensible AI framework for scientific software

Innovation

• LLM-powered system with RAG and reranking
• Custom tools for PETSc knowledge access
• Extensible AI framework for scientific software