TCS-BENCH: Benchmarking State-of-the-Art Generative AI Theoretical Computer Science Research Ability

2026-08-10

AI Summary

This work addresses the lack of systematic evaluation benchmarks for assessing large language models’ ability to generate research-level theorem proofs in theoretical computer science (TCS). It introduces TCS-Bench, the first structured benchmark specifically designed for proof generation of theorems drawn from top-tier conference papers in STOC, FOCS, and SODA. To support self-contained proof synthesis, the benchmark adopts a context-augmented task formulation. A key contribution is the development of a high-precision automated verification agent, which achieves over 90% accuracy on expert-annotated data and effectively aligns with human judgment. This framework provides a reliable and scalable foundation for evaluating the reasoning and formal proof capabilities of generative AI systems in theoretical computer science.

Abstract

We introduce TCS-Bench, a benchmark for evaluating Large Language Models (LLMs) on research-level Theoretical Computer Science (TCS) proof generation. TCS-Bench consists of theorem-proving tasks from papers published at top theoretical computer science venues (STOC, FOCS, and SODA). Each task provides the necessary context to derive a self-contained proof for a target result. We evaluate state-of-the-art models on this benchmark. We verify the correctness of generated proofs via a verification agent, and further benchmark the verifier against human-expert proof judgements on a set of target statements and generated proofs pairs. Our reference verifier achieves over 90% accuracy on the expert labeled set.

Problem

• Large Language Models
• Theoretical Computer Science
• Theorem Proving
• Benchmarking
• Proof Generation

Innovation

• TCS-Bench
• Large Language Models
• Theoretical Computer Science
• Proof Generation
• Verification Agent