Lila

A unified benchmark for math reasoning.

original.zip contains the data used in the Lila paper. We include it for reproducibility. lila.zip contains the data for the official Lila benchmark, which has undergone quality improvements since the paper.

You will need git-lfs to clone these, or you can download them manually (e.g., lila.zip > download).

Contact matthewf@allenai.org with questions.

Do you have a math reasoning dataset that you would like to contribute? Open an issue, or email matthewf@allenai.org and we will work with you to incorporate it!

Cite this dataset and the source datasets (see sources.bib).

@INPROCEEDINGS{Mishra2022Lila,
  author = {
    Swaroop Mishra 
      and Matthew Finlayson 
      and Pan Lu 
      and Leonard Tang 
      and Sean Welleck 
      and Chitta Baral 
      and Tanmay Rajpurohit 
      and Oyvind Tafjord 
      and Ashish Sabharwal 
      and Peter Clark 
      and Ashwin Kalyan},
  title = {Lila: A Unified Benchmark for Mathematical Reasoning},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2022}
}