English
Related papers

Related papers: miniF2F-Lean Revisited: Reviewing Limitations and …

200 papers

Formal specifications, such as pre- and post-conditions provide a solid basis for performing thorough program verification. However, developers rarely provide such formal specifications, hence if AI could help in constructing them, it would…

Software Engineering · Computer Science 2026-03-19 I. S. W. B. Prasetya , Fitsum Kifetew , Davide Prandi

This position paper provides a critical but constructive discussion of current practices in benchmarking and evaluative practices in the field of formal reasoning and automated theorem proving. We take the position that open code, open…

Artificial Intelligence · Computer Science 2025-07-08 Roozbeh Yousefzadeh , Xuenan Cao

We introduce our Leanabell-Prover-V2, a 7B large language models (LLMs) that can produce formal theorem proofs in Lean 4, with verifier-integrated Long Chain-of-Thoughts (CoT). Following our previous work Leanabell-Prover-V1, we continual…

Artificial Intelligence · Computer Science 2025-07-14 Xingguang Ji , Yahui Liu , Qi Wang , Jingyuan Zhang , Yang Yue , Rui Shi , Chenxi Sun , Fuzheng Zhang , Guorui Zhou , Kun Gai

In this work, we conduct an experiment using state-of-the-art LLMs to translate MiniF2F into Rocq. The translation task focuses on generating a Rocq theorem based on three sources: a natural language description, the Lean formalization, and…

Logic in Computer Science · Computer Science 2025-11-25 Jules Viennot , Guillaume Baudart , Emilio Jesùs Gallego Arias , Marc Lelarge

We present FormalProofBench, a private benchmark designed to evaluate whether AI models can produce formally verified mathematical proofs at the graduate level. Each task pairs a natural-language problem with a Lean~4 formal statement, and…

Artificial Intelligence · Computer Science 2026-03-31 Nikil Ravi , Kexing Ying , Vasilii Nesterov , Rayan Krishnan , Elif Uskuplu , Bingyu Xia , Janitha Aswedige , Langston Nashold

Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, leading to their adoption in high-stakes domains such as healthcare, law, and scientific research. However, their reasoning often contains subtle logical…

Software Engineering · Computer Science 2025-12-30 Xinyi Zheng , Ningke Li , Xiaokun Luan , Kailong Wang , Ling Shi , Meng Sun , Haoyu Wang

We introduce Kimina-Prover Preview, a large language model that pioneers a novel reasoning-driven exploration paradigm for formal theorem proving, as showcased in this preview release. Trained with a large-scale reinforcement learning…

As automated reasoning systems advance rapidly, there is a growing need for research-level formal mathematical problems to accurately evaluate their capabilities. To address this, we present Formal Conjectures, an evolving benchmark of…

Translating natural language mathematical statements into formal, executable code is a fundamental challenge in automated theorem proving. While prior work has focused on generation and compilation success, little attention has been paid to…

Large Language Models (LLMs) demonstrate impressive mathematical reasoning abilities, but their solutions frequently contain errors that cannot be automatically checked. Formal theorem proving systems such as Lean 4 offer automated…

Artificial Intelligence · Computer Science 2026-03-18 Sumanth Varambally , Thomas Voice , Yanchao Sun , Zhifeng Chen , Rose Yu , Ke Ye

Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic systems avoid. To bridge this gap, we introduce a formal logic…

Machine Learning · Computer Science 2026-02-02 Chuxue Cao , Jinluan Yang , Haoran Li , Kunhao Pan , Zijian Zhao , Zhengyu Chen , Yuchen Tian , Lijun Wu , Conghui He , Sirui Han , Yike Guo

Training a task-specific small reasoning model is challenging when direct human supervision or high-quality labels are scarce. However, LLMs with reasoning capabilities produce abundant intermediate reasoning traces that can be…

Computation and Language · Computer Science 2025-09-19 Sumanta Bhattacharyya , Sara Riazi , Pedram Rooshenas

Chain-of-Thought (CoT) prompting has become the de facto method to elicit reasoning capabilities from large language models (LLMs). However, to mitigate hallucinations in CoT that are notoriously difficult to detect, current methods such as…

Computation and Language · Computer Science 2025-06-06 Chengwu Liu , Ye Yuan , Yichun Yin , Yan Xu , Xin Xu , Zaoyu Chen , Yasheng Wang , Lifeng Shang , Qun Liu , Ming Zhang

Advances in training, post-training, and inference-time methods have enabled frontier reasoning models to win gold medals in math competitions and settle challenging open problems. Gaining trust in the responses of these models requires…

Machine Learning · Computer Science 2026-04-06 Aaditya Naik , Guruprerana Shabadi , Rajeev Alur , Mayur Naik

Automating the formalization of mathematical statements for theorem proving remains a major challenge for Large Language Models (LLMs). LLMs struggle to identify and utilize the prerequisite mathematical knowledge and its corresponding…

Artificial Intelligence · Computer Science 2026-04-08 Meiru Zhang , Philipp Borchert , Milan Gritta , Gerasimos Lampouras

Recent advances in large language models (LLMs) have demonstrated impressive capabilities in formal theorem proving, particularly on contest-based mathematical benchmarks like the IMO. However, these contests do not reflect the depth,…

Machine Learning · Computer Science 2026-03-10 Jiedong Jiang , Wanyi He , Yuefeng Wang , Guoxiong Gao , Yongle Hu , Jingting Wang , Nailin Guan , Peihao Wu , Chunbo Dai , Liang Xiao , Bin Dong

Reliable autoformalization remains challenging even in the era of large language models (LLMs). The scarcity of high-quality training data is a major bottleneck. Expert annotation requires substantial time and deep expertise in both…

Artificial Intelligence · Computer Science 2026-03-12 Param Biyani , Shashank Kirtania , Yasharth Bajpai , Sumit Gulwani , Ashish Tiwari

Large language models (LLMs) have recently achieved remarkable success in generating rigorous mathematical proofs, with "AI for Math" emerging as a vibrant field of research (Ju et al., 2026). While these models have mastered…

Artificial Intelligence · Computer Science 2026-03-10 Lve Meng , Weilong Zhao , Yanzhi Zhang , Haoxiang Guan , Jiyan He

The aim of Logic2Text is to generate controllable and faithful texts conditioned on tables and logical forms, which not only requires a deep understanding of the tables and logical forms, but also warrants symbolic reasoning over the…

Computation and Language · Computer Science 2022-10-18 Chengyuan Liu , Leilei Gan , Kun Kuang , Fei Wu

Recent advances in automated theorem proving (ATP) through LLMs have highlighted the potential of formal reasoning with Lean 4 codes. However, ATP has not yet be revolutionized by the recent posttraining scaling as demonstrated by Open AI…

Artificial Intelligence · Computer Science 2025-07-15 Jingyuan Zhang , Qi Wang , Xingguang Ji , Yahui Liu , Yang Yue , Fuzheng Zhang , Di Zhang , Guorui Zhou , Kun Gai