diff --git a/README.md b/README.md index 857bea91..f50ca2ba 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,142 @@ # Pytorch A forklet of Pytorch for my own quirky needs. +* Build notes and scripts for different machines I admin. +* CPU builds. +* Meh ROCm AMD GPU builds. +* Proprietary Nvidia builds. +* Flailing at getting larger GPUs + Pytorch on ppc64le going. + # Install -thusly. +thusly, on Debian stable (bookworm/12). + + +# Dependencies +Perhaps this and more: + +## OS +``` +sudo apt install git build-essential libssl-dev zlib1g-dev \ + libbz2-dev libreadline-dev libsqlite3-dev curl \ + libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev \ + libffi-dev liblzma-dev \ + python3-virtualenv python3-pip +``` + +## Python +At present, seems latest Python that works happily with most Pytorch +applications is Python 3.10. + +Use pyenv to manage versions, install something like: + +``` +# :) +curl https://pyenv.run | bash +``` + +Add to `~/.bashrc`, then re-source it (logout/in or whatever): + +``` +export PYENV_ROOT="$HOME/.pyenv" +command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH" +eval "$(pyenv init -)" +eval "$(pyenv virtualenv-init -)" +``` + +Get Python version 3.10: + +``` +pyenv install 3.10 +``` + +Perhaps other versions, ala: + +``` +pyenv install 3.9 +pyenv install 3.11.6 +pyenv install 3.12 +pyenv install 3.13-dev +``` + +## Setup Pip Download Cache +So as to not download files from the Internet multiple times from +each machine in the cluster, a Pip download cacche can be set up +thusly, assuming server IP 192.168.100.101, username debian: + +``` +mkdir -p ~/devel/devpi +cd ~/devel/devpi +virtualenv env +source env/bin/activate +pip install -U setuptools wheel pip +pip install devpi-server devpi-web + +sudo mkdir /srv/devpi +sudo chown debian:debian /srv/devpi + +devpi-init \ + --serverdir /srv/devpi + +devpi-gen-config \ + --host=0.0.0.0 \ + --port 4040 \ + --serverdir /srv/devpi \ + --absolute-urls + +sudo apt install nginx + +sudo cp ~debian/devel/devpi/gen-config/nginx-devpi.conf /etc/nginx/sites-available/ + +cd /etc/nginx/sites-enabled +sudo ln -s ../sites-available/nginx-devpi.conf . + +sudo apt install supervisor +sudo cp ~debian/devel/devpi/gen-config/supervisor-devpi.conf /etc/supervisor/conf.d/ + +crontab -e +@reboot /usr/local/sbin/supervisord -c /home/debian/etc/supervisor-devpi.conf + +supervisord -c gen-config/supervisord.conf + +sudo reboot + +devpi use http://192.168.100.101:4040 + +devpi login root --password '' +devpi user -m root password=FOO +devpi user -l +devpi logoff +devpi user -c debian password=BAR email=devpi@localhost +devpi login debian --password=BAR + +devpi index -c dev bases=root/pypi +devpi use debian/dev +devpi install pytest +``` + + +## Add Pip Download Cache +Add this to clients to use the cache: + +``` +mkdir -p ~/.config/pip +cat > ~/.config/pip/pip.conf <