Korean, Edit

Installing and Understanding AlphaGeometry

Recommended post: 【Algorithm】 Algorithm & Machine Learning Index

1. Overview

2. Installation Process

a. IMO Geometry Problem Solving

b. Natural Language Processing and Large Language Models

1. Overview

⑴ AlphaGeometry:
A language model published in Nature, implementing symbolic deduction to solve IMO-level geometry problems.

⑵ Meaning of Symbol:
Vectorizing each proposition and placing it in the embedding space.

⑶ Symbolic Deduction in Geometry:
Like Euclid’s Elements, creating new propositions starting from axioms about lines, constructions, concentric circles, etc.

⑷ With given problem conditions, searching within the search space for new true propositions to prove the given proposition.

⑸ This process is likened to how a person finds the solution after long contemplation of a math problem.

⑹ Appears to be developed further from FunSearch, a model generating hypotheses made by Google DeepMind (ref).

2. Installation Process

⑴ The procedure in the official documentation causes errors, so follow these steps instead.

⑵ Step 1. Execute the following on an Ubuntu server: The author uses Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-105-generic x86_64).

sudo apt install python3-virtualenv
conda create -n alphageometry python=3.10.9
conda activate alphageometry
git clone https://github.com/google-deepmind/alphageometry.git
cd alphageometry
pip install array-record==0.4.1
pip install clu==0.0.7
pip install seqio==0.0.18
pip install seqio-nightly==0.0.17.dev20231013 
pip install t5==0.9.4
pip install tensorflow-datasets==4.9.3
pip install mesh-tensorflow[transformer]==0.1.21
pip install tfds-nightly==4.9.2.dev202308090034

⑶ Step 2. Remove the following installation items from requirements.txt:

① array-record

② clu

③ seqio

④ seqio-nightly

⑤ t5

⑥ tensorflow-datasets

⑦ mesh-tensorflow[transformer]

⑧ tfds-nightly

⑷ Step 3. bash run.sh: Although this code line skips much of steps 1 and 2, it causes package errors.

① An error occurs at gdown --folder https://bit.ly/alphageometry when executing the code.

⑸ Step 4. Download checkpoint files from the following links and place them in the ag_ckpt_vocab folder:

① https://drive.google.com/uc?id=1qXkmmgoJ8oTYJdFV1xw0xGPpQj6SyOYA

② https://drive.google.com/uc?id=1t-r3KfU8aDbS1UHpdyM3LH21rwSCIXTz

③ https://drive.google.com/uc?id=1mRd6J0UkeWoFUjeVB7BQi5lVNLvPBe31

⑹ Step 5. Execute the remaining code that could not be run in bash run.sh.

MELIAD_PATH=meliad_lib/meliad
mkdir -p $MELIAD_PATH
git clone https://github.com/google-research/meliad $MELIAD_PATH
export PYTHONPATH=$PYTHONPATH:$MELIAD_PATH

DDAR_ARGS=(
  --defs_file=$(pwd)/defs.txt \
  --rules_file=$(pwd)/rules.txt \
);

BATCH_SIZE=2
BEAM_SIZE=2
DEPTH=2

SEARCH_ARGS=(
  --beam_size=$BEAM_SIZE
  --search_depth=$DEPTH
)

LM_ARGS=(
  --ckpt_path=$DATA \
  --vocab_path=$DATA/geometry.757.model \
  --gin_search_paths=$MELIAD_PATH/transformer/configs \
  --gin_file=base_htrans.gin \
  --gin_file=size/medium_150M.gin \
  --gin_file=options/positions_t5.gin \
  --gin_file=options/lr_cosine_decay.gin \
  --gin_file=options/seq_1024_nocache.gin \
  --gin_file=geometry_150M_generate.gin \
  --gin_param=DecoderOnlyLanguageModelGenerate.output_token_losses=True \
  --gin_param=TransformerTaskConfig.batch_size=$BATCH_SIZE \
  --gin_param=TransformerTaskConfig.sequence_length=128 \
  --gin_param=Trainer.restore_state_variables=False
);

echo $PYTHONPATH

⑺ Step 6. Installation is complete, but some errors remain, requiring the installation of additional packages.

pip install matplotlib

sudo apt-get install xvfb
sudo apt-get install python3-tk
Xvfb :1 -screen 0 1024x768x16 &
export DISPLAY=:1.0

pip uninstall flax jax jaxlib
pip install flax==0.5.3 jax==0.4.6 jaxlib==0.4.6

⑻ Step 7. AlphaGeometry can now be executed in ddar mode.

python -m alphageometry \
--alsologtostderr \
--problems_file=$(pwd)/imo_ag_30.txt \
--problem_name=translated_imo_2000_p1 \
--mode=ddar \
"./defs.txt"

⑼ Step 8. To run in alphageometry mode, move the following .gin files to ./:

① Move ./meliad_lib/meliad/transformer/configs/base_htrans.gin to ./.

② Move ./meliad_lib/meliad/transformer/configs/memory_configuration.gin to ./.

③ Move ./meliad_lib/meliad/transformer/configs/trainer_configuration.gin to ./.

⑽ Step 9. AlphaGeometry can now be executed in alphageometry mode.

python -m alphageometry \
--alsologtostderr \
--problems_file=$(pwd)/imo_ag_30.txt \
--problem_name=translated_imo_2000_p6 \
--mode=alphageometry \
"./defs.txt" \
"${SEARCH_ARGS[@]}" \
"${LM_ARGS[@]}"

① Currently unable to execute due to TypeError: JAX only supports number and bool dtypes, got dtype bfloat16 in astype error.

Input: 2024.05.05 22:23