# PyTorch
PyTorch is an alternative to TensorFlow.

If using a KVM, be sure CPU type is set to `host`.

## Get Source
Get PyTorch source code:

```
# This is about 1 gig:
git clone --recursive https://github.com/pytorch/pytorch
```

The recursive git repos contain a mix of permissive licenses, mostly
BSD, MIT, Apache style. No GPL. License owners are mostly Google
and Facebook, with a mix of many others.

## Build Py
Build from scratch with free software options.
PyTorch has a `CMakeLists.txt`, so lets see how it goes with a `cmake` build....

Install build dependencies:
```
apt install cmake cmake-curses-gui g++ python-yaml python-typing
# Note, it uses python-yaml, not python3-yaml...

# Optional deps:
apt install doxygen
apt install libfftw3-dev
apt install libgmp3-dev
apt install libmpfr-dev
apt install libmkldnn-dev
apt install libnuma-dev # Nope, not right one
# Optional deps for BLAS OpenBLAS
apt install libopenblas-dev
# OpenCV
apt install libopencv-dev
# pybind?
apt install pybind11-dev
 pybind11_INCLUDE_DIR             /usr/include/pybind11                                                                                           

# BLAS Eigen
apt install libeigen3-dev # fail?
# ffmpeg
apt install libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavresample-dev libavutil-dev libpostproc-dev libswresample-dev libswscale-dev ffmpeg
# Thusly:
 USE_FFMPEG ON
 FFMPEG_AVCODEC_INCLUDE_DIR       /usr/include/x86_64-linux-gnu/libavcodec                                                                        
 FFMPEG_LIBAVCODEC                /usr/lib/x86_64-linux-gnu                                                                                       
 FFMPEG_LIBAVFORMAT               /usr/lib/x86_64-linux-gnu                                                                                       
 FFMPEG_LIBAVUTIL                 /usr/lib/x86_64-linux-gnu                                                                                       
 FFMPEG_LIBSWSCALE                /usr/lib/x86_64-linux-gnu                                                                                       
# or?
 FFMPEG_AVCODEC_INCLUDE_DIR       /usr/include/x86_64-linux-gnu/libavcodec                                                                        
 FFMPEG_LIBAVCODEC                /usr/lib/x86_64-linux-gnu/libavcodec.so                                                                         
 FFMPEG_LIBAVFORMAT               /usr/lib/x86_64-linux-gnu/libavformat.so                                                                        
 FFMPEG_LIBAVUTIL                 /usr/lib/x86_64-linux-gnu/libavutil.so                                                                          
 FFMPEG_LIBSWSCALE                /usr/lib/x86_64-linux-gnu/libswscale.so                                                                         


# XXX
TORCH_BUILD_VERSION  default is 1.1.0

mkdir build
cd build
cmake ..
ccmake ..
# build takes ~30 minutes
make -j8
```

```
#cmake note
  Generated cmake files are only fully tested if one builds with system glog,
   gflags, and protobuf.  Other settings may generate files that are not well
   tested.

# so maybe:
apt install libgflags-dev
apt install libprotobuf-dev
# glog?
apt install libgoogle-glog-dev # ???
```

Or:

```
python setup.py install
```

How docs say:

```bash
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py build --cmake-only
ccmake build  # or cmake-gui build
```


Notable build options:

```
BLAS --- has multiple options:
         MKL (non-free bits possible)
         vecLib (?)
         FLAME (?)
         Eigen (in Debian)
         ATLAS (in Debian?)
         OpenBLAS (in Debian)
```

More options:

```
BUILD_BINARY ON
BUILD_PYTHON ON
OPENMP_FOUND ON
USE_CUDA OFF
USE_CUDNN OFF
USE_DISTRIBUTED ON
USE_FFMPEG ON
USE_MKLDNN ON  ?? with Debian's packages ?
USE_MKLDNN_CBLAS NO ?
USE_LEVELDB ?? Available in debian.
USE_LMDB ?? Available in Debian.
USE_MPI ON heh
USE_NCCL OFF  nvidia
USE_NUMA ON  ?
USE_NUMPY ON
USE_OBSERVERS ON ?
USE_OPENCL ON ?
USE_OPENCV ON ?
USE_OPENMP ON ?
USE_REDIS ?
USE_ROCKSDB Available in Debian, like leveldb and lmdb
USE_ZMQ Available in Debian, messaging.
USE_ZSTD Available in Debian, compression.
WITH_BLAS ?
WITH_OPENMP ON
CAFFE2_USE_MSVC_STATIC_RUNTIME OFF ??
BUILD_CAFFE2_MOBILE OFF
```

# More misc...
```
git checkout v1.4.0
apt install python3-dev python3-numpy python-numpy
# uh
apt install libcaffe-cpu-dev
```

Seems to be using not python3??


# Misc
```
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
```

# Build PyTorch `pip`

Lets get old Python 2 out of here:

```
apt autoremove --purge python2 python2-minimal python2.7 python2.7-minimal libpython2.7 libpython2.7-minimal libpython2.7-stdlib
git checkout v1.4.0
apt install python3-pip python3-setuptools
# Docs recommend this, but mkl is proprietary:
pip3 install --user numpy ninja pyyaml mkl mkl-include setuptools cmake cffi
# Try:
pip3 install --user --upgrade pip
# Set that new pip in your path, ~/.local/bin/ in ~/.bashrc:
export PATH="~/.local/bin:/usr/lib/ccache:$PATH"

pip3 install --user cmake
pip3 install --user numpy ninja pyyaml setuptools cmake cffi
# From upstream docs:
pip install torch==1.4.0+cpu torchvision==0.5.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
```

# Build with Python Setup

```
git submodule sync
git submodule update --init --recursive
python3 setup.py build --cmake-only
ccmake build
python3 setup.py install --user
```

# Proprietary Bits
Parts to avoid in the PyTorch ecosystem:

* Anaconda is a package manager for Python.The Anaconda repository contains non-free software,
so don't use it.

* "MKL" (Intel's non-free binaries).

* Note, these MKL packages are in Debian: `libmkldnn0 libmkldnn-dev libmkldnn-doc`

* Intel compiler.