NeurIPS 2023 - The Neural MMO Challenge
Train and Submit from Colab
A step-by-step guide for setting up your Colab
kyoung_whan_choe
Open in Colab doesn't seem to work. Click this url instead: https://colab.research.google.com/drive/1v4B5h3MANw6PWH4U4oKba5hfYK5DtlVU
Set up your instance - gpu and google drive¶
In [ ]:
# Check if (NVIDIA) GPU is available
import torch
assert torch.cuda.is_available, "CUDA gpu not available"
In [ ]:
# Set up the work directory
import os
assert os.path.exists("/content/drive/MyDrive"), "Google Drive not mounted"
work_dir = "/content/drive/MyDrive/nmmo/"
Train your agent¶
Install nmmo env and pufferlib¶
In [ ]:
# Install nmmo env and pufferlib
!pip install nmmo pufferlib > /dev/null
!pip show nmmo # should be 2.0.3
!pip show pufferlib # should be 0.4.3
Install the baselines¶
In [ ]:
# Create the work directory, download the baselines code
%mkdir $work_dir
%cd $work_dir
!git clone https://github.com/neuralmmo/baselines --depth=1
In [ ]:
# Install libs to run the baselines
%cd $work_dir
%cd baselines
# Create a requirements_colab.txt
with open(work_dir+'baselines/requirements_colab.txt', "w") as f:
f.write("""
accelerate==0.21.0
bitsandbytes==0.41.1
dash==2.11.1
openelm
pandas
plotly==5.15.0
psutil==5.9.3
ray==2.6.1
scikit-learn==1.3.0
tensorboard==2.11.2
tiktoken==0.4.0
torch
transformers==4.31.0
wandb==0.13.7
""")
!pip install -r requirements_colab.txt > /dev/null
Run python train.py¶
In [ ]:
# Just to check if the training flow works. The checkpoints are saved under nmmo/runs
%cd $work_dir
%cd baselines
ckpt_dir = work_dir + "runs"
!python train.py --runs-dir $ckpt_dir --local-mode true
Submit your checkpoint¶
See https://gitlab.aicrowd.com/Mudou/start-kit
- Sign up for AICrowd and click Participate on the competition page.
- Generate your SSH key and paste it to https://gitlab.aicrowd.com/-/profile/keys
- Clone the stark-kit repository:
git@gitlab.aicrowd.com:Mudou/start-kit.git. HTTP will not work unless you have 2FA configured. - Install the requirements with pip install -r requirements.txt
Set up your SSH connection¶
In [ ]:
# Generating the ssh key with your aicrowd email
# WARNING: Having your ssh key way is not secure, so you should limit using this key for others
my_email = "choe.kyoung@gmail.com" # YOUR AICROWD EMAIL
# See the top for the work_dir, which should be in your google drive
ssh_dir = work_dir + "ssh_key/"
key_file = ssh_dir + "id_rsa"
%cd $work_dir
!mkdir $ssh_dir
!ssh-keygen -t rsa -b 4096 -C $my_email -f $key_file
In [ ]:
# Copy the below text that starts with ssh-rsa to https://gitlab.aicrowd.com/-/profile/keys
ssh_dir = work_dir + "ssh_key/"
key_file = ssh_dir + "id_rsa"
!cat {key_file}.pub
In [ ]:
# Copy the key to default ssh key path - you should see id_rsa
!mkdir /root/.ssh
!cp {key_file}* /root/.ssh
!ls /root/.ssh
!chmod 700 /root/.ssh
# Add the git server as a ssh known host
!touch /root/.ssh/known_hosts
!ssh-keyscan gitlab.aicrowd.com >> /root/.ssh/known_hosts
!chmod 644 /root/.ssh/known_hosts
# You should see something like: Welcome to GitLab, @kyoung_whan_choe!
# to clone the repo and submit
!ssh -T git@gitlab.aicrowd.com
Prepare for submission with the start-kit¶
In [ ]:
# Clone the submission kit repo
%cd $work_dir
!git clone git@gitlab.aicrowd.com:Mudou/start-kit.git
In [ ]:
%cd $work_dir
%cd start-kit/
# Fix permissions
!chmod +x .git/hooks/*
# Install requirements
!pip install -r requirements.txt > /dev/null
In [42]:
# Edit the aicrowd.json -- INCLUDE YOUR NAME
with open(work_dir+'start-kit/aicrowd.json', "w") as f:
f.write("""
{
"challenge_id" : "neurips-2023-the-neural-mmo-challenge",
"authors" : ["kyoung_whan_choe"],
"description" : "Submitting baselines from the submission tutorial colab, take 17"
}
""")
Submit!¶
In [ ]:
# Click the link to authenticate into aicrowd
%cd $work_dir
%cd start-kit/
!python tool.py submit "track1-submission-tutorial-17"
If submission hangs or got an git lfs error ...¶
In [ ]:
# If submission hangs or you get a git lfs error, retry after running:
!git lfs fetch --all origin
!git lfs push --all aicrowd
Content
Comments
You must login before you can post a comment.