FormalProofBench: Can Models Write Graduate Level Math Proofs That Are Formally Verified?

1. 1Design a private benchmark with natural-language problems and Lean~4 formal statements
2. 2Pair each problem with a Lean proof accepted by the Lean 4 checker
3. 3Evaluate AI models' ability to output valid Lean proofs
4. 4Assess model performance on advanced undergraduate and graduate mathematics problems

Researchers and developers in AI and mathematics can benefit from FormalProofBench to assess and improve model performance, while educators can use it to enhance graduate-level math education

💡 AI models can be evaluated on their ability to produce formally verified mathematical proofs at the graduate level using FormalProofBench

📝 Can AI models write formally verified grad-level math proofs? 🤔