notes on pytorch building

master
ml server 2020-02-08 12:36:30 -07:00
parent 2cb732eac7
commit d1edfd4ace
1 changed files with 85 additions and 1 deletions

View File

@ -1,19 +1,102 @@
# PyTorch
PyTorch is an alternative to TensorFlow.
If using a KVM, be sure CPU type is set to `host`.
## Get Source
Get PyTorch source code:
```
# This is about 1 gig:
git clone --recursive https://github.com/pytorch/pytorch
```
The recursive git repos contain a mix of permissive licenses, mostly
BSD, MIT, Apache style. No GPL. License owners are mostly Google
and Facebook, with a mix of many others.
## Build Py
Build from scratch with free software options.
PyTorch has a `CMakeLists.txt`, so lets see how it goes with a `cmake` build....
Install build dependencies:
```
apt install cmake
apt install cmake cmake-curses-gui g++ python-yaml python-typing
# Note, it uses python-yaml, not python3-yaml...
# Optional deps:
apt install doxygen
apt install libfftw3-dev
apt install libgmp3-dev
apt install libmpfr-dev
apt install libmkldnn-dev
apt install libnuma-dev # Nope, not right one
# Optional deps for BLAS OpenBLAS
apt install libopenblas-dev
# OpenCV
apt install libopencv-dev
# pybind?
apt install pybind11-dev
pybind11_INCLUDE_DIR /usr/include/pybind11
# BLAS Eigen
apt install libeigen3-dev # fail?
# ffmpeg
apt install libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavresample-dev libavutil-dev libpostproc-dev libswresample-dev libswscale-dev ffmpeg
# Thusly:
USE_FFMPEG ON
FFMPEG_AVCODEC_INCLUDE_DIR /usr/include/x86_64-linux-gnu/libavcodec
FFMPEG_LIBAVCODEC /usr/lib/x86_64-linux-gnu
FFMPEG_LIBAVFORMAT /usr/lib/x86_64-linux-gnu
FFMPEG_LIBAVUTIL /usr/lib/x86_64-linux-gnu
FFMPEG_LIBSWSCALE /usr/lib/x86_64-linux-gnu
# or?
FFMPEG_AVCODEC_INCLUDE_DIR /usr/include/x86_64-linux-gnu/libavcodec
FFMPEG_LIBAVCODEC /usr/lib/x86_64-linux-gnu/libavcodec.so
FFMPEG_LIBAVFORMAT /usr/lib/x86_64-linux-gnu/libavformat.so
FFMPEG_LIBAVUTIL /usr/lib/x86_64-linux-gnu/libavutil.so
FFMPEG_LIBSWSCALE /usr/lib/x86_64-linux-gnu/libswscale.so
# XXX
TORCH_BUILD_VERSION default is 1.1.0
mkdir build
cd build
cmake ..
ccmake ..
```
```
#cmake note
Generated cmake files are only fully tested if one builds with system glog,
gflags, and protobuf. Other settings may generate files that are not well
tested.
# so maybe:
apt install libgflags-dev
apt install libprotobuf-dev
# glog?
apt install libgoogle-glog-dev # ???
```
Or:
```
python setup.py install
```
How docs say:
```bash
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py build --cmake-only
ccmake build # or cmake-gui build
```
```
Notable build options:
```
@ -51,6 +134,7 @@ USE_ZSTD Available in Debian, compression.
WITH_BLAS ?
WITH_OPENMP ON
CAFFE2_USE_MSVC_STATIC_RUNTIME OFF ??
BUILD_CAFFE2_MOBILE OFF
```