Deep Learning for Mathematical Reasoning (DL4MATH)

This repository is the reading list on Deep Learning for Mathematical Reasoning (DL4MATH).

Contributors: Pan Lu @UCLA, Liang Qiu @UCLA, Wenhao Yu @Notre Dame, Sean Welleck @UW, Kai-Wei Chang @UCLA

For more details, please refer to the paper: A Survey of Deep Learning for Mathematical Reasoning.

:bell: If you have any suggestions or notice something we missed, please don't hesitate to let us know. You can directly email Pan Lu (lupantech@gmail.com), comment on the twitter, or post an issue on this repo.

🧰 Resources

Related Surveys

• A Survey of Question Answering for Math and Science Problem, arXiv:1705.04530 [paper]
• The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers, TPAMI 2019 [paper]
• Representing Numbers in NLP: a Survey and a Vision, NACL 2021 [paper]
• Survey on Mathematical Word Problem Solving Using Natural Language Processing, ICIICT 2021 [paper]
• A Survey in Mathematical Language Processing, arXiv:2205.15231 [paper]
• Partial Differential Equations Meet Deep Neural Networks: A Survey, arXiv:2211.05567 [paper]
• :fire: Reasoning with Language Model Prompting: A Survey, arXiv:2212.09597 [paper]
• :fire: Towards Reasoning in Large Language Models: arXiv:2212.10403 [paper]
• :fire: A Survey for In-context Learning, arXiv:2301.00234 [paper]

Related Blogs

• :fire: How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources, Dec 2022, Yao Fu’s Notion [link]

Workshops

• :fire: The 1st MATH-AI Workshop: the Role of Mathematical Reasoning in General Artificial Intelligence, ICLR 2021 [website]
• :fire: Math AI for Education: Bridging the Gap Between Research and Smart Education (MATHAI4ED), NeurIPS 2021 [website]
• :fire: The 1st Workshop on Mathematical Natural Language Processing, EMNLP 2022 [website]
• :fire: The 2nd MATH-AI Workshop: Toward Human-Level Mathematical Reasoning, NeurIPS 2022 [website]
• :fire: FLAIM: Formal Languages, AI and Mathematics, IHP & META 2022 [YouTube]
• :fire: AI to Assist Mathematical Reasoning: A Workshop, NASEM 2023 [YouTube]

Talks

• Can GPT-3 do math? | Grant Sanderson and Lex Fridman, 2020 [YouTube]
• Computer Scientist Explains One Concept in 5 Levels of Difficulty, 2022 [YouTube]

🎨 Mathematical Reasoning Benchmarks

Math Word Problems (MWP)

• [AI2/Verb395] Learning to Solve Arithmetic Word Problems with Verb Categorization, EMNLP 2014 [paper]
• [Alg514] Learning to automatically solve algebra word problems, ACL 2014 [paper]
• [IL] Reasoning about Quantities in Natural Language, TACL 2015 [paper]
• [SingleEQ] Parsing Algebraic Word Problems into Equations, TACL 2015 [paper]
• [DRAW] Draw: A challenging and diverse algebra word problem set, 2015 [paper]
• [Dolphin1878] Automatically solving number word problems by semantic parsing and reasoning, EMNLP 2015 [paper]
• [Dolphin18K] How well do computers solve math word problems? large-scale dataset construction and evaluation, ACL 2016 [paper]
• [MAWPS] MAWPS: A math word problem repository, NAACL-HLT 2016 [paper]
• [AllArith] Unit dependency graph and its application to arithmetic word problem solving, AAAI 2017 [paper]
• [DRAW-1K] Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problems, ACL 2017 [paper]
• :fire: [Math23K] Deep neural solver for math word problems, EMNLP 2017 [paper]
• [AQuA] Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems, ACL 2017 [paper]
• [Aggregate] Mapping to Declarative Knowledge for Word Problem Solving, TACL 2018 [paper]
• :fire: [MathQA] MathQA: Towards interpretable math word problem solving with operation-based formalisms, NAACL-HLT 2019 [paper]
• [ASDiv] A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers, ACL 2020 [paper]
• [HMWP] Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems, EMNLP 2020 [paper]
• [Ape210K] Ape210k: A large-scale and template-rich dataset of math word problems, arXiv:2009.11506 [paper]
• :fire: [SVAMP] Are NLP Models really able to Solve Simple Math Word Problems?, NAACL-HIT 2021 [paper]
• :fire: [GSM8K] Training verifiers to solve math word problems, arXiv:2110.14168 [paper]
• :fire: [IconQA] IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning, NeurIPS 2021] [paper]
• :fire: [MathQA-Python] Program synthesis with large language models, arXiv:2108.07732 [paper]
• [ArMATH] ArMATH: a Dataset for Solving Arabic Math Word Problems, LREC 2022 [paper]
• :fire: [TabMWP] Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning, arXiv:2209.14610, 2022 [paper]

Theorem Proving (TP)

• [MML] Four Decades of Mizar, Journal of Automated Reasoning 2015, [paper]
• [HolStep] HolStep: A Machine Learning Dataset for Higher-order Logic Theorem Proving, ICLR 2017 [paper]
• [GamePad] GamePad: A Learning Environment for Theorem Proving, ICLR 2019 [paper]
• :fire: [CoqGym] Learning to Prove Theorems via Interacting with Proof Assistants, ICML 2019 [paper]
• [HOList] HOList: An environment for machine learning of higher order logic theorem proving, ICML 2019 [paper]
• [IsarStep] IsarStep: a Benchmark for High-level Mathematical Reasoning, ICLR 2021 [paper]
• [LISA] LISA: Language models of ISAbelle proofs, AITP 2021 [paper]
• [INT] INT: An Inequality Benchmark for Evaluating Generalization in Theorem Proving, ICLR 2021 [paper]
• :fire: [NaturalProofs] NaturalProofs: Mathematical Theorem Proving in Natural Language, NeurIPS 2021 [paper]
• [NaturalProofs-Gen] NaturalProver: Grounded Mathematical Proof Generation with Language Models, NeurIPS 2022 [paper]
• :fire: [MiniF2F] MiniF2F: a cross-system benchmark for formal Olympiad-level mathematics, ICLR 2022 [paper]
• :fire: [LeanStep] Proof Artifact Co-training for Theorem Proving with Language Models, ICLR 2022 [paper]
• :fire: [miniF2F+informal] Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs, arXiv:2210.12283 [paper]

Geometry Problem Solving (GPS)

• :fire: [GEOS] Solving geometry problems: Combining text and diagram interpretation, EMNLP 2015 [paper]
• [GeoShader] Synthesis of solutions for shaded area geometry problems, The Thirtieth International Flairs Conference, 2017 [paper]
• [GEOS++] From textbooks to knowledge: A case study in harvesting axiomatic knowledge from textbooks to solve geometry problems, EMNLP 2017 [paper]
• [GEOS-OS] Learning to solve geometry problems from natural language demonstrations in textbooks, Proceedings of the 6th Joint Conference on Lexical and Computational Semantics, 2017 [[paper](https://aclanthology.org