135 lines
3.0 KiB
Markdown
135 lines
3.0 KiB
Markdown
# TensorFlow Serving
|
|
HOWTO Set up and run TensorFlow Serving.
|
|
This is to this particular configuration.
|
|
|
|
# Software
|
|
Main software in use:
|
|
|
|
* Debian
|
|
* Proxmox
|
|
* Ceph
|
|
* Python 3
|
|
* TensorFlow Serving
|
|
|
|
# Installation
|
|
Install TensorFlow Serving. The recommended way is using `docker`, but
|
|
here we build from source.
|
|
|
|
* https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md
|
|
|
|
* https://github.com/tensorflow/serving/blob/master/tensorflow_serving/tools/docker/Dockerfile.devel
|
|
|
|
|
|
|
|
## Install Bazel
|
|
Instead of building Bazel, download the version that TensorFlow Serving builds with,
|
|
currently version `1.2.1` as shown in the docker builder.
|
|
|
|
Install Bazel dependencies:
|
|
|
|
```
|
|
# Note, this installs python 2.7....
|
|
apt install g++ zlib1g-dev unzip python
|
|
```
|
|
|
|
Install bazel .deb from releases here.
|
|
* firefox https://github.com/bazelbuild/bazel/releases
|
|
|
|
Note get the version Serving wants, not the latest release.
|
|
|
|
```
|
|
wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb
|
|
wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb.sha256
|
|
dpkg -i bazel_1.2.1-linux-x86_64.deb
|
|
apt -f install
|
|
```
|
|
|
|
## Install Dependencies
|
|
Dependencies. Note, there are likely fewer dependencies than listed
|
|
in the docs, since `bazel` is installed from `.deb`, not built.
|
|
|
|
```
|
|
apt update
|
|
apt install --no-install-recommends \
|
|
automake \
|
|
build-essential \
|
|
ca-certificates \
|
|
curl \
|
|
git \
|
|
libcurl3-dev \
|
|
libfreetype6-dev \
|
|
libpng-dev \
|
|
libtool \
|
|
libzmq3-dev \
|
|
mlocate \
|
|
pkg-config \
|
|
python-dev \
|
|
software-properties-common \
|
|
swig \
|
|
unzip \
|
|
wget \
|
|
zip \
|
|
zlib1g-dev \
|
|
python3-distutils
|
|
```
|
|
|
|
Not installed:
|
|
```
|
|
openjdk-8-jdk \
|
|
openjdk-8-jre-headless \
|
|
```
|
|
|
|
|
|
## Compile TensorFlow Serving
|
|
HOWTO compile TensorFlow Serving:
|
|
|
|
```
|
|
git clone https://github.com/tensorflow/serving
|
|
cd serving
|
|
|
|
git checkout 2.1.0
|
|
|
|
bazel build --color=yes --curses=yes \
|
|
${TF_SERVING_BAZEL_OPTIONS} \
|
|
--verbose_failures \
|
|
--output_filter=DONT_MATCH_ANYTHING \
|
|
${TF_SERVING_BUILD_OPTIONS} \
|
|
tensorflow_serving/model_servers:tensorflow_model_server
|
|
```
|
|
|
|
Build `pip` package:
|
|
|
|
```
|
|
bazel build --color=yes --curses=yes \
|
|
${TF_SERVING_BAZEL_OPTIONS} \
|
|
--verbose_failures \
|
|
--output_filter=DONT_MATCH_ANYTHING \
|
|
${TF_SERVING_BUILD_OPTIONS} \
|
|
tensorflow_serving/tools/pip_package:build_pip_package
|
|
|
|
|
|
bazel-bin/tensorflow_serving/tools/pip_package/build_pip_package \
|
|
/tmp/pip
|
|
```
|
|
|
|
# Install TensorFlow Server with `pip`
|
|
Install with `pip`, don't use bazel, or build, etc.
|
|
|
|
```
|
|
# Set PATH, add to ~/.bashrc
|
|
export PATH="~/.local/bin:/usr/lib/ccache:$PATH"
|
|
. ~/.bashrc
|
|
sudo apt install python3-pip
|
|
pip3 install --upgrade --user pip
|
|
# Should return the one in ~/.local:
|
|
which pip3
|
|
# Install it, needs ~3 gigs free in /tmp
|
|
pip3 install --user tensorflow-serving-api
|
|
```
|
|
|
|
# Misc
|
|
See also:
|
|
|
|
* https://github.com/tobegit3hub/simple_tensorflow_serving
|
|
|