Python setup noted
parent
d547174905
commit
652f59b7d9
137
README.md
137
README.md
|
@ -1,9 +1,142 @@
|
|||
# Pytorch
|
||||
A forklet of Pytorch for my own quirky needs.
|
||||
|
||||
* Build notes and scripts for different machines I admin.
|
||||
* CPU builds.
|
||||
* Meh ROCm AMD GPU builds.
|
||||
* Proprietary Nvidia builds.
|
||||
* Flailing at getting larger GPUs + Pytorch on ppc64le going.
|
||||
|
||||
|
||||
# Install
|
||||
thusly.
|
||||
thusly, on Debian stable (bookworm/12).
|
||||
|
||||
|
||||
# Dependencies
|
||||
Perhaps this and more:
|
||||
|
||||
## OS
|
||||
```
|
||||
sudo apt install git build-essential libssl-dev zlib1g-dev \
|
||||
libbz2-dev libreadline-dev libsqlite3-dev curl \
|
||||
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev \
|
||||
libffi-dev liblzma-dev \
|
||||
python3-virtualenv python3-pip
|
||||
```
|
||||
|
||||
## Python
|
||||
At present, seems latest Python that works happily with most Pytorch
|
||||
applications is Python 3.10.
|
||||
|
||||
Use pyenv to manage versions, install something like:
|
||||
|
||||
```
|
||||
# :)
|
||||
curl https://pyenv.run | bash
|
||||
```
|
||||
|
||||
Add to `~/.bashrc`, then re-source it (logout/in or whatever):
|
||||
|
||||
```
|
||||
export PYENV_ROOT="$HOME/.pyenv"
|
||||
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
|
||||
eval "$(pyenv init -)"
|
||||
eval "$(pyenv virtualenv-init -)"
|
||||
```
|
||||
|
||||
Get Python version 3.10:
|
||||
|
||||
```
|
||||
pyenv install 3.10
|
||||
```
|
||||
|
||||
Perhaps other versions, ala:
|
||||
|
||||
```
|
||||
pyenv install 3.9
|
||||
pyenv install 3.11.6
|
||||
pyenv install 3.12
|
||||
pyenv install 3.13-dev
|
||||
```
|
||||
|
||||
## Setup Pip Download Cache
|
||||
So as to not download files from the Internet multiple times from
|
||||
each machine in the cluster, a Pip download cacche can be set up
|
||||
thusly, assuming server IP 192.168.100.101, username debian:
|
||||
|
||||
```
|
||||
mkdir -p ~/devel/devpi
|
||||
cd ~/devel/devpi
|
||||
virtualenv env
|
||||
source env/bin/activate
|
||||
pip install -U setuptools wheel pip
|
||||
pip install devpi-server devpi-web
|
||||
|
||||
sudo mkdir /srv/devpi
|
||||
sudo chown debian:debian /srv/devpi
|
||||
|
||||
devpi-init \
|
||||
--serverdir /srv/devpi
|
||||
|
||||
devpi-gen-config \
|
||||
--host=0.0.0.0 \
|
||||
--port 4040 \
|
||||
--serverdir /srv/devpi \
|
||||
--absolute-urls
|
||||
|
||||
sudo apt install nginx
|
||||
|
||||
sudo cp ~debian/devel/devpi/gen-config/nginx-devpi.conf /etc/nginx/sites-available/
|
||||
|
||||
cd /etc/nginx/sites-enabled
|
||||
sudo ln -s ../sites-available/nginx-devpi.conf .
|
||||
|
||||
sudo apt install supervisor
|
||||
sudo cp ~debian/devel/devpi/gen-config/supervisor-devpi.conf /etc/supervisor/conf.d/
|
||||
|
||||
crontab -e
|
||||
@reboot /usr/local/sbin/supervisord -c /home/debian/etc/supervisor-devpi.conf
|
||||
|
||||
supervisord -c gen-config/supervisord.conf
|
||||
|
||||
sudo reboot
|
||||
|
||||
devpi use http://192.168.100.101:4040
|
||||
|
||||
devpi login root --password ''
|
||||
devpi user -m root password=FOO
|
||||
devpi user -l
|
||||
devpi logoff
|
||||
devpi user -c debian password=BAR email=devpi@localhost
|
||||
devpi login debian --password=BAR
|
||||
|
||||
devpi index -c dev bases=root/pypi
|
||||
devpi use debian/dev
|
||||
devpi install pytest
|
||||
```
|
||||
|
||||
|
||||
## Add Pip Download Cache
|
||||
Add this to clients to use the cache:
|
||||
|
||||
```
|
||||
mkdir -p ~/.config/pip
|
||||
cat > ~/.config/pip/pip.conf <<EOF
|
||||
[global]
|
||||
trusted-host = 192.168.100.101
|
||||
index-url = http://192.168.100.101:4040/root/pypi/+simple/
|
||||
|
||||
[search]
|
||||
index = http://192.168.100.101:4040/root/pypi/
|
||||
EOF
|
||||
```
|
||||
|
||||
|
||||
# Get Deepcrayon Pytorch Repo
|
||||
```
|
||||
git clone https://spacecruft.org/deepcrayon/pytorch
|
||||
cd pytorch/
|
||||
```
|
||||
|
||||
|
||||
# Upstream
|
||||
|
@ -11,6 +144,8 @@ Main upstream:
|
|||
|
||||
* https://github.com/pytorch/pytorch
|
||||
|
||||
See also: `README-upstream.md`.
|
||||
|
||||
|
||||
# Disclaimer
|
||||
I am not a programmer.
|
||||
|
|
Loading…
Reference in New Issue