HGR

Code for "Hologram Reasoning for Solving Algebra Problems with Geometry Diagrams"

Requirements

Python 3.8
torch 2.3.0
python3-pip

Create a conda virtual environment:

conda create -n HGR python=3.8
conda activate HGR

Install all required Python dependencies:

pip3 install -r requirements.txt

Prepare VF3 for graph pattern matching:

cd vf3

For Linux/MacOS:

make vf3.so

For Windows:

make vf3.dll

Prepare the Dataset

First, download the Geometry3K dataset from Google Drive. Save train.zip, val.zip, test.zip and logic_forms.zip into db folder. Then, unzip data files:

sh unzip_data.sh

Run HGR

In HGR, two strategies are implemented to solve problems:

1. Heuristic Strategy:

   • This strategy involves exhaustively searching through all available graph models. While straightforward, this strategy can become inefficient as the complexity and number of graph models increase.

2. Model Selection Agent:

   • To improve efficiency, HGR employs a model selection agent that uses deep reinforcement learning (DQN). This agent selects the most appropriate graph model at each reasoning step, significantly reducing the search space and improving performance in complex scenarios.

If you want to display the reasoning process, please comment out the line os.environ['ENVIRONMENT'] = 'production' in the reasoner/config.py file. This will allow the program to run in development mode, providing more detailed output of the reasoning steps and debugging information.

However, for batch testing, it is recommended to keep the production setting to ensure optimal performance and minimal logging.

The Heuristic Strategy.

Test a single question using the heuristic strategy:

python eval.py --use_annotated --question_id {ID}

Test all questions in the test set using the heuristic strategy:

python eval.py --use_annotated

eval.py generates an eval.log file in the output folder, which logs the results of each question.

Model Selection Agent

For training the model selection agent, a crucial step is the initialization of the memory, which stores the samples used to train the agent. These samples are generated using two strategies:

1. Heuristic Strategy:
   • This strategy is employed to generate model selection samples that are more likely to have positive rewards.
2. Random Selection:
   • In contrast, the random selection strategy generates samples more likely to result in negative rewards.

Prepare the initial memory

python train.py --init_memory

Pre-train the model selection agent on the initial memory:

python train.py --pre_train

Train the model selection agent:

python train.py --batch_size {BATCH_SIZE} --gamma {GAMMA} --beam_size {BEAM_SIZE} --lr {LR}

Test a single question using the agent:

python eval.py --use_annotated --use_agent --question_id {ID}

Test all questions in the test set using the agent:

python eval.py --use_annotated --use_agent

Calculate Accuracies

You can obtain accuracies for different question types using the logfile generated by eval.py:

python sub_acc.py --log_file_path {LOG_FILE_PATH}

Citation

If the paper or the code helps you, please cite the paper in the following format :

@misc{huang2024hologram,
      title={Hologram Reasoning for Solving Algebra Problems with Geometry Diagrams}, 
      author={Litian Huang and Xinguo Yu and Feng Xiong and Bin He and Shengbing Tang and Jiawen Fu},
      year={2024},
      eprint={2408.10592},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2408.10592}, 
}